We Have To Talk About Flask

Posted by
on under

Flask 3.0 was released on September 30th, 2023, along with a parallel 3.0 release of Werkzeug, its main dependency. That day, the Flask-Login extension, one of the most popular of all Flask extensions, stopped working due to a backwards incompatible change introduced in Werkzeug. It is October 19th when I'm writing this, and Flask-Login remains broken. As a result, any person using my Flask Mega-Tutorial will hit issues, because my tutorial uses Flask-Login. Not only that, every Flask tutorial that features Flask-Login, from every author, in every language, in written or video form, is going to fail for as long as this problem remains. Hard to believe, right? (Update: a fixed release of Flask-Login was published on October 30th)

If this was the first occurrence of something of this nature in the Flask community, I would hope it would serve as a lesson for the Flask maintainers to learn from and avoid in the future. Sadly, this happens pretty much every time there is a major release of Flask, and sometimes minor ones too. Why does this happen? How can it be avoided? In this article I'll try to make an assessment of the current situation and how it can be prevented going forward.

There is now an update to this post as well.

What Happened with Flask-Login?

In case you have not been affected by this, let me show you what the problem is. Let's go ahead and install Flask-Login:

$ pip install flask-login
...
Installing collected packages: MarkupSafe, itsdangerous, click, blinker, Werkzeug, Jinja2, Flask, flask-login
Successfully installed Flask-3.0.0 Jinja2-3.1.2 MarkupSafe-2.1.3 Werkzeug-3.0.0 blinker-1.6.3 click-8.1.7 flask-login-0.6.2 itsdangerous-2.1.2

You can see here that Flask-Login installed version 0.6.2, which PyPI reports with a release date of July 25, 2022. A few dependencies were imported as well, because I did this in a brand new virtual environment. Among them, the new Flask 3.0.0 and Werkzeug 3.0.0, which are dated September 30th, 2023, also according to PyPI.

So far so good. Let's try to import Flask-Login in a brand new Python session:

>>> import flask_login
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/miguel/venv/lib/python3.12/site-packages/flask_login/__init__.py", line 12, in <module>
    from .login_manager import LoginManager
  File "/Users/miguel/venv/lib/python3.12/site-packages/flask_login/login_manager.py", line 33, in <module>
    from .utils import _create_identifier
  File "/Users/miguel/venv/lib/python3.12/site-packages/flask_login/utils.py", line 14, in <module>
    from werkzeug.urls import url_decode
ImportError: cannot import name 'url_decode' from 'werkzeug.urls' (/Users/miguel/venv/lib/python3.12/site-packages/werkzeug/urls.py). Did you mean: 'urlencode'?

So here is the problem. Flask-Login fails to import because it wants this url_decode() function from the werkzeug.urls module that does not appear to exist.

How big of a deal is this? Let me put it this way. How many Flask books and tutorials can we estimate that exist out there? I'd risk that at least we are talking about a number in the thousands, if not more. I'm going to guess that a large part of these tutorials use Flask-Login. Every person trying to learn Flask with one of these since September 30th, 2023 and until a still unknown date later than today (October 19th) when this is finally addressed, is going to hit a roadblock that they are unlikely to be able to figure out on their own. Maybe they'll think the tutorial they are following is bad or out of date and will switch to a different one, only to get the same error once again. How many people is being affected by this? A few hundred? Thousands? More? I don't really know, but I believe it is a really big number.

The issues board for Flask-Login does not show any open issues about this. We can change the filters in the issue board to look at closed issues instead, and bingo, I counted 21 closed issues from the last couple of weeks where people complain about this. I found 3 pull requests intended to address this issue. Two were rejected, and one was merged on October 2nd. So the issue has actually been fixed, and what's missing is just a release with the fix to be pushed to PyPI, the Python Package Index.

Why is a release being delayed? I'm unclear on how many people have access to publish releases of this package. It appears that the original creator of Flask-Login isn't interested in maintaining the package anymore, and another person to whom he may have given access to the project is currently on vacation, but intends to get a release out when back. So in the end, this appears to be a bus factor issue more than anything else.

The Flask team added a deprecation warning for this function in the previous release of Werkzeug, so they feel the maintainers of Flask-Login had time to address the issue ahead of the function's removal.

But the Flask-Login team is not actively developing the extension anymore. Flask-Login is a mature library that does what it does well, so there hasn't been a need to make changes to it in recent years. The creator of the extension has probably moved on to other projects and maybe isn't using the extension anymore and is certainly not standing by and checking for deprecation messages from Flask to appear.

Blaming Flask-Login does not help when you consider that this url_decode() function that was removed from Werkzeug is used by an uncountable number of projects besides Flask-Login (including some of my own). We know that Flask-Login is likely going to be fixed within days or weeks at the worst, at which point we'll all forget this. But the large number of applications that use this function directly will hit this problem whenever they upgrade Flask, and this can be in the next week or in 5 years. I guarantee you that for years to come, reports of this failed import from Werkzeug are going to continue popping up in Stack Overflow and other developer forums.

This is probably going to get me some enemies, but I think the Flask developers are more responsible for this disaster than the Flask-Login team is. This isn't the first time new Flask releases break extensions or content. In fact, I have come to expect that every Flask release will require me to rush fixes for some of my packages or my content. Sometimes even minor releases manage to break some of my things.

Backwards Compatibility

The current issue with Flask-Login and the many similar ones from the past are all issues of backwards compatibility. New versions of Flask and/or Werkzeug introduce significant changes, so extensions and tutorials that were designed for older versions start to fail and need to be updated so that they continue to work.

Every time I discuss backwards compatibility, what comes to mind are the contrasting views that Microsoft and Apple hold on the topic.

Microsoft has always gone to great lengths to ensure backwards compatibility of their products, especially their Windows operating system, and also their XBox gaming console. They considered it a great asset that people would confidently upgrade their OS or console, knowing that all the applications and games that they owned would continue to work as before. Not sure if this still goes on, but Microsoft used to have entire teams dedicated to testing software on soon to be released versions of Windows, and create patches in the OS to ensure the software continued to work without needing an update from the software vendor. Of course all this effort is unknown to most people, even though it was tremendously successful. Why? Because the success metric for this work is actually very boring, the goal is that nothing breaks and users can continue to use their software through upgrades. And of course this doesn't make big news.

Apple has a different stance. They believe that maintaining old stuff is too much of a burden, so from time to time they introduce major changes and innovations to their products, usually with limited transition options. Their Macintosh line of computers was initially based on the Motorola 68000 processors. In 1994 they decided to migrate to a PowerPC architecture. In 2005 they migrated again, this time to Intel CPUs. Finally in 2020 they started yet another platform migration to their own Apple Silicon, based on the ARM architecture. Interestingly enough, people (and especially developers) love them, in spite of these disruptive changes.

Who do you align with? I honestly do not see these two views as opposing, and can appreciate the good in both.

The Microsoft thinking that the software and games that you use every day are the most important part of your computer or console, and that the operating system is just there in a minor supporting role for them greatly resonates with me.

But I also see how from time to time having a hard reset allows Apple to introduce innovation and freshness into their product lines. Apple users have come to expect these changes and willingly tolerate some inconvenience during the transition period in exchange for significant improvements in efficiency, performance, style or a combination of them.

Flask and Backwards Compatibility

So who do you think the Flask team aligns with in terms of backwards compatibility?

They do not align with Microsoft for sure! But do they align with Apple? I actually do not think so.

Let's look at the specific error that appeared in Flask-Login. The extension tried to import the url_decode() function from Werkzeug. Where has this function gone? I did some digging, and can present you with its complete history:

  • In releases prior to 1.0.0, the function was imported with from werkzeug import url_decode
  • In release 1.0.0 (February 2020), the function was relocated and the import changed to from werkzeug.urls import url_decode
  • In release 2.3.0 (April 2023), a deprecation message was added when the function was used
  • In release 3.0.0 (September 2023), the function was removed, with the idea that people should migrate to a similar function available in the Python standard library.

So as you see, the function did not evolve or become better (at least not in any significant way). It just moved around a couple of times until the day it was removed.

I mentioned above that this isn't the first time that Flask releases cause breakages. In fact, I have seen this pattern repeat time and time again. There are two recent instances that I'm going to mention, just so that you don't think I'm exaggerating. The first one is related to how to configure Flask's debug vs. production modes using an environment variable:

  • In versions prior to Flask 1.0, the FLASK_DEBUG environment variable was used. A value of 1 configured debug mode. A value of 0 or undefined disabled debug mode.
  • In version 1.0 (2018), the FLASK_ENV environment variable was introduced to replace FLASK_DEBUG. You would set it to development to enable debug mode, or production to disable debug mode, similar to the NODE_ENV variable used by the Express framework in Node.js. The FLASK_DEBUG variable was not removed nor deprecated, but the documentation and examples encouraged developers to use FLASK_ENV.
  • In version 2.2.0 (2022), the FLASK_ENV environment variable was deprecated and the documentation was reverted to recommend FLASK_DEBUG.
  • In version 2.3.0 (2023), the FLASK_ENV environment variable was removed, leaving FLASK_DEBUG once again as the only environment variable to control debug mode. A circle of life kind of a thing, I guess.

The other instance that affected me personally was related to the get_engine() method of the Flask-SQLAlchemy extension, which is currently maintained by one of the Flask core team members. This is a publicly documented method that survived without any major changes until the 3.0 release, but then this happened:

  • Prior to version 3.0.0, the method signature was get_engine(app=None, bind=None).
  • In version 3.0.0 (October 2022), the method signature was changed to get_engine(bind_key=None). In addition to the removal of one argument and the renaming of the other, the function was deprecated (why would you change a function in a major way and deprecate it at the same time?).
  • In preparation for release 3.1.0 (September 2023), get_engine() was removed, but then the maintainer changed their mind and added it back (My complaining may have had some influence on this..., not sure).
  • The function is now scheduled to be removed in upcoming release 3.2.0, and will be replaced by a property called engines that provides exactly the same information, but as a dictionary.

I think Flask, like Apple, likes to break things without worrying too much about the past.

But unlike Apple, and this is the part I have a lot of trouble with, these refactorings do not give the community anything in exchange! The Flask team constantly reorganizes the code in silly and meaningless ways, forcing extension developers and content creators to adapt just to keep things from breaking, but these changes do not contribute any tangible improvements that justify the effort.

So Flask is definitely not aligned with Apple either, in my view.

Where Do We Go From Here?

This is unfortunately a question I do not have a clear answer for.

It is likely that the Flask core team will continue to refactor and change the code in ways that I'm sure makes sense to them, but that do not provide substantial benefits to the community at large. If this is the case, I will continue to keep up with their changes, as I'm sure most other extension developers and content creators will. One day I may decide to not care anymore, and then one of my projects may end up being at the center of another incident like the Flask-Login one.

But maybe, given the magnitude of the current issue, this is the straw that broke the camel's back, as they say, and from now on Flask core developers will feel pressured to think it twice or three times before they do more gratuitous refactoring.

I surely hope so!

I have written an update to this post with a collection of comments and feedback I have received.

October 31st, 2023 update: Armin Ronacher, creator of Flask and Werkzeug has tweeted in support of my claims. Note that Armin has not been directly involved with these projects in the last few years, so none of my criticism was directed at him or his choices, but instead at the current maintainers.

Become a Patron!

Hello, and thank you for visiting my blog! If you enjoyed this article, please consider supporting my work on this blog on Patreon!

42 comments

  • #1 James said

    I learned how to program using Python for the web by following your tutorial and from the companion book. I still use Flask at work. So this comes from a place of deep appreciation for what you've done, and also for what Flask has made possible for me.

    But maybe the answer is to have the core web engine and the rest of the things you need to make a webapp (like login, databases, etc.) all be part of the same package, and all be managed by the same org, and where maintenance is distributed to a core team with a steering council? If you think I am describing Django, yes, I am describing Django.

  • #2 Jeff said

    I don't follow - are you pinning your version dependencies? A major version is allowed to make breaking changes. That's the whole of a point of a major version?

  • #3 Miguel Grinberg said

    @Jeff: You don't seem to understand my point. I have no problem to support evolving software when it is warranted. My problem is having to constantly revise my content with renames and other refactorings of no importance. Pinning is not really the issue here, when you author a tutorial you have to make sure it works with recent versions, so I can't decide to stay on an older release. I always have to upgrade and follow current versions of the packages I'm teaching.

  • #4 SsaliJonathan said

    Thanks Miguel. I’ve also landed into this error and many people who watch my tutorials also tell me they have landed into it. I thought it was something wrong with the code but I’m glad you clarified it. Thanks Miguel

  • #5 Daniel Demmel said

    I guess the solution is either to convince the Flask maintainer team to offer a more stable API or if they don't want to then vote with your feet and migrate to a different ecosystem? I guess it's hard to gauge how stable the API is over time though, so not easy to pick another.

    That actually gave me an idea for a cool project, documentation sites usually aren't great with keeping versions around and switching between them, so some sort of archive that tracks changes over time and parses the diff (maybe job for a large language model fine tuned on Python?) to show how it evolved would be pretty useful.

  • #6 Miguel Grinberg said

    @Daniel: I think this is much simpler than that. I don't mind breaking changes that help move the Flask project forward. In the same way I put it with some inconveniences while using a Mac M2 coming from Intel. This is fine. My problem is when this is done without a strong reason. The Flask maintainers need to realize that the community is huge, and that no matter how much planning they put into it, a breaking change is going to cause a lot of people pain. I don't want to see the community agonizing over a broken extension for over two weeks because they felt a function they had in Werkzeug duplicated one in the Python standard library. It costs nothing to keep providing access to this function through Werkzeug and then this would have been prevented. Same with silly renames of arguments or environment variables. It's nonsense.

  • #7 Daedox said

    Hey Miguel i totally agree!
    I try to keep in my projects both the used components and also their dependencies actual as possible. So I was directly confronted with the werkzeug 3.0 url_decode() problem in one of my test builds. Of course each of us could use workarounds but it would in chaos very soon.

    The Flask Maintainer, created something awesome, should become aware of their responsibility. Because as you said no one needs useless changes

    Flask Login, your awesome projects and much more, I wont wanna miss them. I just hope that the maintainers of course also with help of community keep running all these great projects as long as possible.

    Thanks for all!

  • #8 Abdur-Rahmaan Janhangeer said

    Being a long-time Flask user, for me the dark point has always been Werkzeug updates!

  • #9 Varoon said

    I their a fix coming? Can I still follow your work? I wanted to continue the Flask Mega tutorial. Thanks

  • #10 Miguel Grinberg said

    @Varoon: this is not something I'm involved with. You can avoid the problem if you use older releases of Flask and Werkzeug, which is not great, but is the most reasonable thing to do until Flask-Login is fixed.

  • #11 Pamphile (tupui) said

    It is likely that the Flask core team will continue to refactor and change the code in ways that I'm sure makes sense to them, but that do not provide substantial benefits to the community at large.

    In your essay, you seem to not really acknowledge the fact that the maintainers need to actually maintain Flask. As a maintainer of SciPy, I can relate to such discussion and I am pretty sure they do not just do things to annoy everyone...

    The best way forward is to get in touch with the maintainers and actively participate in the development (I am not saying that to you specifically, but to everyone relying on Flask). Maintainers can only know what they know and if the community is silent even in RC phases or after a warning has been out for months, then what can they do? Maybe the deprecation policy/cycle is too short? They cannot also be held responsible for the slow response of extension authors. That's an ecosystem issue and making support tiers could be a solution?

    We have similar struggles for SciPy. People only complain loudly sometimes multiple versions down the road. As maintainers, we need to make some calls, and yes we do try to make our lives easier and sometimes even knowing that this will break people's code. Because in the end, years down the road, we have to keep maintaining this and we also need to keep up with OS, trends, people's ideals, etc.

  • #12 Abdur-Rahmaan Janhangeer said

    One last thing is that I think at some point we realize that the solution is a social one. This is why building a strong community is important. If the past ecosystem turned out like this, then the new one should not be like this.

    The Flask Community Workgroup (flaskcwg), which handles FlaskCon, was started because of this. Here is an insightful excerpt from the motivation:

    "... However, there are many Flask-related issues which deserve some attention. The fix to those issues lies not in mere merges or code fixes. One such example is the plugins/extensions ecosystem. A recurrent and commonly observed pattern is that a project rises in popularity and becomes the de-facto reference for a specific task then the maintainer goes offline, cut off from the project. Sometimes other maintainers themselves have no merge right, leaving the project as good as dead.

    Due to many factors, the projects soon become obsolete, less efficient or outright broken. But people still use them sometimes unknowingly exposing their applications to vulnerabilities. One help point from the WG is ... (flaskcwg.github.io)"

  • #13 Miguel Grinberg said

    @Pamphile: I believe I mention in the article that I'm sure they think they have a good reason for making these refactorings, so yes, we agree.

    The best way forward is to get in touch with the maintainers

    Yes, and I have discussed this issue with the maintainers on several occasions, so they know how I feel about these changes.

    if the community is silent

    The community is largely unaware of what happens until something breaks in a big way. If you mean the community of extension developers, then yes, I have been letting them know every time they inflicted pain on me that I considered they could have avoided if they had more careful planning of what is worth deprecating and what isn't.

    Maybe the deprecation policy/cycle is too short?

    For a start I think they should not remove anything on minor releases. The Flask-Login incident happened on a major release, but I mention other issues I experienced which were introduced with minor releases.

    As far as too short or too long I think that is really not the most important consideration. I think for a project that enjoys such large popularity the maintainers should feel some responsibility to prevent API changes unless necessary. Removing some code that another library duplicates is not a good excuse to introduce breaking changes. Going back and forth between FLASK_DEBUG and FLASK_ENV is actually crazy and unacceptable to me.

    They cannot also be held responsible for the slow response of extension authors.

    Why not? It is well known that Flask-Login has been running on fumes for several years. The day the pushed the 3.0 release out they knew Flask-Login wasn't fixed. So I do hold them responsible. They went ahead with the release and decided that having this incident was an acceptable collateral damage. Remember, this change doesn't improve Flask in any way! They just want you to import this function from another place. What is the cost of keeping a wrapper function to the other one in Werkzeug and prevent Flask-Login and a ton of other apps from breaking?

    sometimes even knowing that this will break people's code

    You are talking to me as if I wasn't also a maintainer. I do understand what you are saying, and I'm also constantly debating if I should or should not make a breaking change in my own projects. I accept that this is part of life and we all have to deal with breaking changes. What I do not accept is that the project refactors things without having an important reason. I hate it when I have to introduce breaking changes in my projects, and sometimes I do, but I would never do it just because I don't like the name of a variable.

  • #14 Alexey said

    Vue and Vite both have special “community CI” to prevent their major dependencies from breaking:

    https://github.com/vitejs/vite-ecosystem-ci

    https://github.com/vuejs/ecosystem-ci

    Introducing something like this may help Flask project.

  • #15 Gustavo said

    Actually, Apple has invested a great deal to make sure their changes in architecture, which were massive as you stated, are backwards compatible and painless for their users, inso far as to making a whole emulation system just for that, which was also quite light and efficient (Rosetta for PowerPC to Intel and now Rosetta 2 for Intel to ARM), sometimes even besting performance of native Windows apps in emulation. On iOS, they migrated from 32bit to 64bit basically without anyone noticing, which for Google and Microsoft was a huge hurdle. Also, Microsoft has had to deal with no major changes in architecture at all... Was it Windows 10 that forced you to install the OS anew with no upgrade option?

    I don't think Apple has a problem with backwards compatibility per se, rather, they make changes to move the market where they want knowingly, such as removing the floppy drive, removable batteries, etc., but backwards compatibility in software is a big priority in Apple.

    Anyway, not to make it an Apple discussion but I believe your idea of backwards compatibility on Apple is not correct.

  • #16 Miguel Grinberg said

    @Alexey: I don't disagree, and in fact there is a similar initiative already in existance for Flask. The point I'm trying to get across in this article, however, is that the Flask team has been introducing breaking changes indiscriminately. They have done so in every major and minor release since 2.0, at least. This is unsustainable. I don't want them to stop introducing breaking changes, just to do so more judiciously.

  • #17 Miguel Grinberg said

    @Gustavo: Yes, of course they provide some help during the transition. Giving you an emulator is what I call in the article a "limited transition option". The goal of Rosetta is to give you some time until you are able to replace all your software with new versions that are native to the incoming platform. Neither Rosetta nor Rosetta 2 are intended as a long-term solution.

    Also I would like to point out that from the side of developers at least the transition from PowerPC to Intel was massive and incredibly costly due to the change from big-endian to little-endian, which required way more effort than recompiling with a new compiler.

  • #18 Alex said

    Great essay, couldn’t agree more. I’m a maintainer of Dash, and we eventually added an upper bound on Flask, essentially acknowledging that we expect each new minor to break something in our ecosystem (ironically we’ve actually had more trouble with minors than majors). We did so against significant community pushback (both users wanting immediate access to the latest version and arguments that upper bounds are bad in principle), but also with significant community support as many have shared these frustrations - in our case some users aren’t even aware they’re using Flask until it breaks. Flask and Werkzeug are the only packages we limit this way. We owe an incredible debt to Flask for everything it makes possible, and I know firsthand the struggles of maintaining a library, so in our case we decided to only allow new Flask with Dash after we’ve thoroughly QA’d it.

  • #19 Iris said

    Why aren't we talking about Flask-Login having the wrong dependency version constraints? If it was tested with Flask 2.x, its dependencies should not be so liberal as to include Flask 3.x, which is a potentially backwards-incompatible change.

  • #20 Miguel Grinberg said

    @Iris: I am not talking about Flask-Login having wrong dependency constraints because that doesn't help anything. We can talk all you want about that, but that does not reflect our current reality. There are a lot of things that are not ideal. You can complain, blame someone else and shrug, or you can recognize that the world is imperfect in a lot of ways and still find ways to minimize issues.

    Also, Flask introduces breaking changes pretty much in every release, major or minor. This one time we are in the 3.0.0, but what about the pointless breaking changes they introduced in 2.3.0, 2.2.0, 2.1.0 and older? Feel free to check the Flask and Werkzeug change logs to see all the things the were deleted in those releases. How would an extension protect against deleted code in a minor release?

    The problem here is not Flask-Login. Flask maintainers should stop making trivial refactorings that contribute nothing of substance to the project.

  • #21 Bruno said

    Can someone point me in the right direction to what would be the temporary solution for this?

    Been trying to add an authentication system to my flask app, but it is not working.

    I have seen some solutions online, but cant figure it out.

    Can someone tell me the right versions of Flask, flask_sqlalchemy, werkzeug

    Tried several possibilities, but now am stuck at:

    werkzeug - 2.3.0
    Flask-Login - 0.6.2
    flask - 3.0.0

  • #22 Miguel Grinberg said

    @Bruno: This is exactly the situation that I think the Flask folks should have prevented by being more careful with their deprecations. My suggestion is that you install Flask-Login from the main branch of the GitHub repository, so that you get the fix. With that installed you should be able to use latest Flask and Werkzeug versions. Else, you can continue to wait until Flask-Login releases a fixed release.

  • #23 Bruno Afonso said

    Thanks,

    I installed via pip install:

    pip install git+https://github.com/maxcountryman/flask-login.git

    getting:
    ImportError: cannot import name '_request_ctx_stack' from 'flask'

    colorama 0.4.6
    Flask 3.0.0
    Flask-BabelEx 0.9.4
    Flask-Login 0.7.0
    Flask-Mail 0.9.1
    Flask-Principal 0.4.0
    Flask-Security 3.0.0
    Flask-SQLAlchemy 3.1.1
    Flask-WTF 1.2.1
    Werkzeug 3.0.0

    I think I'll wait to see if they release an update. Don't really know how to solve this.

  • #24 Miguel Grinberg said

    @Bruno: I'm sorry you are going through this, but this isn't my battle to fight, so I cannot help. I merely reported on what I think is a problem in how our community works. If you need help with this you can try the Flask discord or yeah, just wait it out.

  • #25 Chris said

    @Bruno that may be Flask-BabelEx which has been archived for 3 years. If you want Flask 3 then I think you will have to switch to Flask-Babel (I'm hoping anyway I haven't tried it yet). But of course that could mean some work on your code...

Leave a Comment