It's Time For A Change: datetime.utcnow() Is Now Deprecated

This is actually very common in web applications or other types of networking servers, which are configured with UTC time and normalize all dates and times to this timezone when they enter the system. It is also a best practice to store naive datetimes representing UTC in databases.

If you store "naive datetimes representing UTC" in your DB then doesn't that clash with Python's naive datetimes? Python's naive datetimes are usually assumed to represent local time rather than UTC, unless they come from these two funny functions. If all the times in your DB are in UTC then the right type to use in python is a datetime with datetime.UTC attached, not a naive one. Similarly, if you want to convert all times to UTC on the way into your web application as you say is best practice, then once you've done so the result should be a datetime with datetime.UTC attached not a naive one. (It may be the case that the whole webserver is configured to be in UTC time so these are more or less the same, but it's still better to be explicit just like it's better to encode all your text in UTF-8 instead of using the system locale.)

@Alex: I get what you are saying, but the idea that naive datetimes represent local time is fairly recent, and only put to practice in a handful of functions. The Python 3.12 documentation has what I consider the correct definition for naive datetime objects:

A naive object does not contain enough information to unambiguously locate itself relative to other date/time objects. Whether a naive object represents Coordinated Universal Time (UTC), local time, or time in some other timezone is purely up to the program, just like it is up to the program whether a particular number represents metres, miles, or mass. Naive objects are easy to understand and to work with, at the cost of ignoring some aspects of reality.

So there is really no clash. As long as you know that you are working with naive datetimes and only perform operations that do not require knowing the timezone (which in my opinion is a pre-requisite to working with naive datetimes), there is absolutely no problem.

I can turn the argument around and say that it really makes no sense to use a naive datetime to represent a local time, since you can explicitly state that you want local time by storing the local timezone in the object. The footgun exists from both sides of the argument.

Hmmm, I don't know, the very purpose of UTC is to create a reference, so a "naive" datetime should have always defaulted to UTC + 0.

@Jeffry: No, that was never the intention. There is already a way to represent UTC timestamps as aware datetimes, and that works well. A naive datetime does not specify a timezone. This works for applications that do not care about timezones, or for applications that know what timezone they want to use already.

You Python programmers don't know how good you've got it!

I'd love to only have to deal with "problems" like the one you're describing.

About six months ago, I was given some Rust software to maintain, and it has been one of the worst experiences of my professional life. Everything is a disaster, especially anything involving crates. So many of them are unmaintained, and so many of them are rubbish and full of bugs to begin with.

Even if Python has some legacy quirks, at least what's there is pretty darn good to begin with, and it's getting better and better with each new release.

Frankly, I'm about to rewrite this damn Rust software in Python, I'm that frustrated with how awful it is.

The python datetimes where a mess to begin with and have been the culprit of many many many issues. Pytz made serious errors giving rise to Arrow and Pendulum. Naive objects should have never existed at all. "naive_utcnow" is an oxymoron. There is no such thing.

"An application may be designed in such a way that all dates and times are in a single timezone that is known in advance. In this case there is no need for individual datetime instances to carry their own timezones, since this uses more memory and processing power for no benefit, since all these timezones would be the same and it would never be necessary to perform timezone math or conversions."

I worked in Finance where people assumed this for payment systems. The mess that this has become when trying to reconcile when it became needed to a certain point in the future is incredible. DST settings and time zones change quite regularly. Imagine what this would mean for cutoff times. So if you have a naive 'local' datetime object and you saved that to naive UTC how do you get it back to the local time zone? Indeed you need to re-add timezone information.

"This is actually very common in web applications or other types of networking servers, which are configured with UTC time and normalize all dates and times to this timezone when they enter the system. It is also a best practice to store naive datetimes representing UTC in databases. The DateTime type in SQLAlchemy represents a naive datetime object by default, for example. This is such a common database pattern that SQLAlchemy provides a recipe for applications that use aware datetime objects to convert these to and from naive ones on the fly as they are saved to or loaded from the database."

The pattern is to only provide time zone aware datetimes where the user requires them, everywhere else it should be in UTC. So if you have an app in front of your database you can store datetimes in a naive format as long you are absolutely certain all your datetimes get transposed to UTC. Guess what: your databases doesn't do the conversion for you if the field is naive so your application needs to do so.

The problem with naive timezone (as I understand in Python) is that people might use it not knowing what they are really working with, just because it's easy.
But first of all, the name is misleading (as you do not get UTC). There is a obviously a problem when you change the timezone, and there will probably be a problem with DST too : what is the real time point of a naive date between 2 and 3 the date you go back from summer time to winter time?

As you, and the old adage say:

Explicit is better than implicit

I totally agree with you on that one!

The problem is fairly complex, I even made a conference on the timezone subject (and learned a lot preparing it 🤯)

Thank you for your blog post, it does spread the time problem to everyone 👍

Hey Miguel 👋, would you mind to point a resource or link, in case you know, where is explained why python took the design decision to implement naive datetime using local timezone instead of UTC, I'm just curious. Thanks for the amazing and useful article.

It's not true that it's a "fairly recent" change to have naive timestamps represent local time. Might be a difference between Python 2 and 3, but that's ancient times, not recent times.

You can also look at the real world - eg. look at your wall clock, your task bar, or the "departures" board of your local train station. You'll see naive times that you can assume to be local time. This isn't some obscene idea that the Python devs came up with - it's literally how the world works.
It's also e.g. specified in ISO8601 that if no timezone it in a timestamp, then it's "local time".

It's ok though, to have an in-database or in-memory representation that works differently. You can make your own rules for internal representation. But it makes good sense for the Python datetime API to work the way it does, considering that e.g. hour and minute are public attributes, and these make absolutely no sense without timezone information.

@Bolke: I'm confused about the point you are trying to make. I'm not against aware datetimes, you do not have to sell them to me. I also have no problem if you think that naive datetimes are problematic.

I worked in Finance where people assumed this for payment systems.

Okay. So this isn't one of those applications for which naive datetimes are good. I never said naive datetimes should be used by every application. But you seem to be saying that nobody should use naive datetimes?

@Anon: I have no idea. The documentation disagrees with this. Here is the definition of naive datetimes in Python 3.12's docs:

A naive object does not contain enough information to unambiguously locate itself relative to other date/time objects. Whether a naive object represents Coordinated Universal Time (UTC), local time, or time in some other timezone is purely up to the program, just like it is up to the program whether a particular number represents metres, miles, or mass. Naive objects are easy to understand and to work with, at the cost of ignoring some aspects of reality.

The new interpretation of naive datetimes is something that must have grown organically and was never officially documented, at least not yet.

@Jonas:

It's not true that it's a "fairly recent" change to have naive timestamps represent local time.

It's recent enough that this interpretation of what a naive datetime represents hasn't been documented yet. I copied it a few times already, but here it is once again, from the 3.12 documentation:

A naive object does not contain enough information to unambiguously locate itself relative to other date/time objects. Whether a naive object represents Coordinated Universal Time (UTC), local time, or time in some other timezone is purely up to the program, just like it is up to the program whether a particular number represents metres, miles, or mass. Naive objects are easy to understand and to work with, at the cost of ignoring some aspects of reality.

So you can't really say that naive datetimes are intended to be local time when the docs say otherwise, right?

But it makes good sense for the Python datetime API to work the way it does

What do you mean? You do realize that the datetime API does not work in the way you are describing, and not even the documentation agrees with you? The datetime API is a mess with all of its inconsistencies. So I do not really understand why you are putting it as some sort of example. The reality is that we are having these discussions because of how bad the design of this API is.

Hey!

The problem with timezones is that they are not related to time, but to jurisdiction, which is most commonly a country.

UTC is a "jurisdictionless" time zone - no country is assigned UTC as its time zone, as it does not have control over it. This also means having timestamps in UTC is usually a sane thing to do, since all the other time zones are referenced to UTC, and converting from UTC to any time zone is only one step.

Unfortunately, at over 200 contries in the world today, this means there is high probability that one jurisdiction will decide to change its time zone. Thus, you need to update tzinfo regularly.

Even when tzinfo is updated, there are caveats with using UTC. Timestamps are only really valid for past time, not future, as if you have a calendar app and take your inputs in local timestamps, store them as UTC, and if you later display them in an updated local time zone, the hours might mismatch.

I think that most of the use cases are probably better served by timestamps in UTC than anything else.

Modelling time is hard. :D

I have historically used the distinction between naive and aware datetime objects as a means to enforce discipline: with a naive object there is no chance that anyone will forget that you must store/query the location meta-data in order to get/preserve the correct local time. With timezone-aware objects people get sloppy and start assuming that "someone else", "somewhere else" has handled this.

My experience is that if you really care about datetime being both correct and appropriate, just putting timezones on timestamps is not good enough. You need to maintain control and allow overrides at a more detailed level. Countries declare tz-changes with extremely short notice. Important customers (large corporations) decide to simply not accept certain time changes made by foreign governments under certain periods. Airports stay in the same place but change countries/state from one day to another and with that their timezone.

Having spent a lot of time with Python code where we really care about timezones, the only option has always been to use UTC everywhere combined with fine-grained location identifiers. The location identifiers combined with the UTC timestamps are turned into UTC-offsets via lookups at runtime. 99.5% of the time this gives the same result as having just slapped an EST or CET or whatever suffix onto the original datetime object, but those last 0.5% are important to some people, sometimes.

All that said; it will probably be fine with mandatory timezone attribute on datetime objects. One can write linter tools to catch non-UTC timezones being used/imported, if one cares about not having them.

Regardless of how you think naive datetimes should work, or whether there is any written rule about how they work, de facto they are treated like local timestamps by several functions in datetime, and so utcnow and fromutctimestamp don't do what they say. This is extremely confusing even when you already know about it. I consult the docs for datetime probably more than any other module.

So, as Python, you can either deprecate the functions that do not do what they say or intruduce a breaking changes across the entire module to change the de facto behaviour of the thing that has been the de-facto behaviour for at least a decade. Remember the last time Python introduced breaking changes?

Arguably this is all moot to me though because the moment you care about the fact that your times are in UTC, you by definition care about timezone information and should be explicit. Sort of as an aside, people care about timezone information more often than they naively (ha) think they do.

@Nick: I can apply the same argument to your side by swapping utcnow() and fromutctimestamp() with timestamp(). All these functions do something that is unexpected, and all these functions explain what they do that is unexpected in the documentation. The same argument that you are making against storing UTC in naive datetimes can be made against storing local time in naive datetimes.

utcnow and fromutctimestamp don't do what they say

The timestamp method doesn't do what it says either! How can you tell that dt.timestamp() uses local time? That makes no sense either. All these functions do something you don't expect, and they all explain what this is in the documentation.

Naive DTs in non-UTC time zones are a huge footgun, though, due to their interaction with DST and the inability to distinguish timestamps around the jump and the oddness of timedelta math when those show up. I guess they’re fine if you’re in UTC, but you’re vulnerable to really painful edge cases otherwise.

Naive datetimes are important to me, as I have two different applications that need to always reflect the original input to the in user, regardless of what time they are in.

If an event is taking place at a location at 10pm, I can't have this value being adjusted, just because the user is querying from a different time zone. The naive solution works perfectly and is a lot more sensible than saving and juggling timezones to make sure the right data is being delivered.

In SQLAlchemy, I have a lot of timestamp columns that are defaulting to "datetime.utcnow", should I default to "lambda: datetime.now(UTC)" instead now?

I also wonder if this change has other implications in SQLAlchemy

@kion: If you don't mind the deprecation warning then you don't have to change anything. If you want to stop seeing the warning, then yes, change to the timezone-aware format. The database will still record a native timestamp, as far as I recall most if not all databases store naive timestamps.

Clearly it should have been that presenting no timezone defaults to UTC and also having naive timestamps be generated the same in all computers, that is, in UTC.

Maybe the problem with that was simply that "hey let's have another utcnow that actually gives you an utc timestamp" would be too confusing