I've written about blooming weeds before, such as
jewel weed.
Blackberry can be considered a weed, too; it is in bloom in late April / early May in north Georgia; it spreads aggressively; it also sometimes yields good fruit.

Another weed that both blooms and has the potential to spread is thistle. There are a lot of varieties, and I'm not sure which one is in my back yard. I have a couple of plants back there that seem to recur, although I cut them before they set seed. Most thistles are annual or biennial, so I'm not sure why this happens every spring in the same place. One that is at least somewhat perennial and can spread by roots is Canadian thistle.
That notwithstanding, the blooms are large, pretty, and have a nice fragrance. If they set seed, though, watch out. one plant can produce tens of thousands of seeds per year.
There are open fields not too far away that are heavily-infested and unattended, so they're getting worse with every passing year.

Here's another view of the plant that shows its form better; Canadian thistles have smooth upper stems, so that checks. In the background of both shots you can see another non-native invasive that is truly worthless, privet. I have a lot of it around and I'm going to try to cut a lot of it back this year. While it does bloom, I don't really like those flowers at all. Bizarrely, privet is still sold in nurseries...
Shown below is a stand of thistle near an Interstate on ramp. Although the display may be reasonably attractive in the photo, if it isn't mowed soon a million seeds will be produced within a couple of months (possibly quite literally a million--or more).
While one or two plants can be nice, I'm fortunate I don't have more of it.
Update: Okay, so maybe it's not a good idea to have even one or two. About three weeks after the above pictures were taken, the first flower had turned to seed, although new blooms were still opening. I didn't get to it and bag it before some seeds escaped...but not many. Alas.