JSON Web Tokens with Public Key Signatures

JSON Web Tokens offer a simple and powerful way to generate tokens for APIs. These tokens carry a payload that is cryptographically signed. While the payload itself is not encrypted, the signature protects it against tampering. In their most common format, a "secret key" is used in the generation and verification of the signature. In this article I'm going to show you a less known mechanism to generate JWTs that have signatures that can be verified without having access to the secret key.

Quick Introduction to JSON Web Tokens (JWTs)

In case you are not familiar with JWTs, let me first show you how to work with them using Python with the pyjwt package. Create a virtual environment, and install pyjwt in it:

(venv) $ pip install pyjwt

Now let's say you want to create a token that gives a user with id 123 access to your application. After you verify that the user has provided the correct username and password, you can generate a token for the user:

>>> import jwt
>>> secret_key = "a random, long, sequence of characters that only the server knows"
>>> token = jwt.encode({'user_id': 123}, secret_key, algorithm='HS256')
>>> token
'eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ1c2VyX2lkIjoxMjN9.oF_jJKavmWrM6d_io5M5PBiK9AKMf_OcK4xpc17kvwI'

The jwt.encode() function has three arguments of which the most important is the first, containing the token payload. This is the information that you want stored in the token. You can use anything that can be serialized to a JSON dictionary as a payload. The payload is where you record any information that identifies the user. In the simplest case this is just the user id like in the example above, but you can include other user information such as a username, user roles, permissions, etc. Here is a more complex token:

>>> token = jwt.encode({
...     'user_id': 123,
...     'username': 'susan',
...     'roles': ['user', 'moderator']
... }, secret_key, algorithm='HS256')
>>> token
'eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ1c2VyX2lkIjoxMjMsInVzZXJuYW1lIjoic3VzYW4iLCJyb2xlcyI6WyJ1c2VyIiwibW9kZXJhdG9yIl19.fRZ4ButrxgElKB57TlunFo4bGGHponRJcV54NMF-hgM'

As you can see, the more data you write in the payload, the longer the token is, because all that data is physically stored in the token. By looking at the resulting JWTs you may think that the data that you put in the tokens is encrypted, but this is actually incorrect. You should never write sensitive data in a JWT, because there is no encryption. This seemingly random sequence of characters that you see in these tokens is just generated with a simple base64 encoding.

In addition to user information, the payload of a JWT can include a few fields that apply to the token itself, and have a predefined meaning. The most useful of these is the exp field, which defines an expiration time for the token. The following example gives the token a validity period of 5 minutes (300 seconds):

>>> from time import time
>>> token = jwt.encode({
...     'user_id': 123,
...     'username': 'susan',
...     'roles': ['user', 'moderator'],
...     'exp': time() + 300
... }, secret_key, algorithm='HS256')
>>> token
'eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ1c2VyX2lkIjoxMjMsInVzZXJuYW1lIjoic3VzYW4iLCJyb2xlcyI6WyJ1c2VyIiwibW9kZXJhdG9yIl0sImV4cCI6MTUyODU2MDc3My41Mzg2ODkxfQ.LuicSWptAYHBXKJnM3iz9V07Xz_vSKb3AheYXOC444A'

Other predefined fields that can be included in the JWT are nbf (not before), which defines a point in time in the future at which the token becomes valid, iss (issuer), aud (audience) and iat (issued at). Consult the JWT specification if you want to learn more about these.

The second argument to jwt.encode() is the secret key. This is a string that is used in the algorithm that generates the cryptographic signature for the token. The idea is that this key must be known only to the application, because anyone who is in possession of this key can generate new tokens with valid signatures. In a Flask or Django application, you can pass the configured SECRET_KEY for this argument.

The last argument in the jwt.encode() call is the signing algorithm. Most applications use the HS256 algorithm, which is short for HMAC-SHA256. The signing algorithm is what protects the payload of the JWT against tampering.

The value returned by jwt.encode() is a byte sequence with the token. You can see in all the above examples that I decoded the token into a UTF-8 string, because a string is easier to handle.

Once your application generates a token it must return it to the user, and from then on, the user can authenticate by passing the token back to the server, which prevents the user from having to constantly send stronger credentials such as username and password. Using JWTs for authentication is considered more secure than usernames and passwords, because you can set an appropriate expiration time, and in that way limit the damage that can be caused in the case of a leak.

When the application receives a JWT from the user it needs to make sure that it is a legitimate token that was generated by the application itself, which requires generating a new signature for the payload and making sure it matches the signature included with the token. Using the first of the example tokens above, this is how the verification step is done with pyjwt:

>>> import jwt
>>> secret_key = "a random, long, sequence of characters that only the server knows"
>>> token = 'eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ1c2VyX2lkIjoxMjN9.oF_jJKavmWrM6d_io5M5PBiK9AKMf_OcK4xpc17kvwI'
>>> payload = jwt.decode(token, secret_key, algorithms=['HS256'])
>>> payload
{'user_id': 123}

The jwt.decode() call also takes three arguments: the JWT token, the signing key, and the accepted signature algorithms. Note how in this call a list of algorithms is provided, since the application may want to accept tokens generated with more than one signing algorithm. Note that while the algorithms argument is currently optional in pyjwt, there are potential vulnerabilities that can occur if you don't pass the list of algorithms explicitly. If you have applications that call jwt.decode() and don't pass this argument, I strongly advise you to add this argument.

The return value of the jwt.decode() call is the payload that is stored in the token as a dictionary ready to be used. If this function returns, it means that the token was determined to be valid, so the information in the payload can be trusted as legitimate.

Let's try to decode the token from above that had an associated expiration time. I have generated that token more than five minutes ago, so even though it is a valid token, it is now rejected because it has expired:

>>> token = 'eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ1c2VyX2lkIjoxMjMsInVzZXJuYW1lIjoic3VzYW4iLCJyb2xlcyI6WyJ1c2VyIiwibW9kZXJhdG9yIl0sImV4cCI6MTUyODU2MDc3My41Mzg2ODkxfQ.LuicSWptAYHBXKJnM3iz9V07Xz_vSKb3AheYXOC444A'
>>> payload = jwt.decode(token, secret_key, algorithms=['HS256'])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/migu7781/Documents/dev/flask/jwt-examples/venv/lib/python3.6/site-packages/jwt/api_jwt.py", line 105, in decode
    self._validate_claims(payload, merged_options, **kwargs)
  File "/Users/migu7781/Documents/dev/flask/jwt-examples/venv/lib/python3.6/site-packages/jwt/api_jwt.py", line 135, in _validate_claims
    self._validate_exp(payload, now, leeway)
  File "/Users/migu7781/Documents/dev/flask/jwt-examples/venv/lib/python3.6/site-packages/jwt/api_jwt.py", line 176, in _validate_exp
    raise ExpiredSignatureError('Signature has expired')
jwt.exceptions.ExpiredSignatureError: Signature has expired

It is also interesting to see what happens if I take one of the tokens above, make a change to any of the characters in the string and then try to decode it:

>>> token = 'eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ1c2VyX2lkIjoxMjN9.oF_jJKavmWrM6d_io5M5PBiK9AKMf_OcK4xpc17kvwO'
>>> payload = jwt.decode(token, secret_key, algorithms=['HS256'])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/miguel/jwt/venv/lib/python3.6/site-packages/jwt/api_jwt.py", line 93, in decode
    jwt, key=key, algorithms=algorithms, options=options, **kwargs
  File "/home/miguel/jwt/venv/lib/python3.6/site-packages/jwt/api_jws.py", line 157, in decode
    key, algorithms)
  File "/home/miguel/jwt/venv/lib/python3.6/site-packages/jwt/api_jws.py", line 224, in _verify_signature
    raise InvalidSignatureError('Signature verification failed')
jwt.exceptions.InvalidSignatureError: Signature verification failed

So as you see, if jwt.decode() returns back a dictionary, you can be sure that the data in that dictionary is legitimate and can be trusted (at least as much as you are sure your secret key is really secret).

Using Public-Key Signatures with JWTs

A disadvantage of the popular HS256 signing algorithm is that the secret key needs to be accessible both when generating and validating tokens. For a monolithic application this isn't so much of a problem, but if you have a distributed system built out of multiple services running independently of each other, you basically have to choose between two really bad options:

• You can opt to have a dedicated service for token generation and verification. Any services that receive a token from a client need to make a call into the authentication service to have the token verified. For busy systems this creates a performance bottleneck on the authentication service.
• You can configure the secret key into all the services that receive tokens from clients, so that they can verify the tokens without having to make a call to the authentication service. But having the secret key in multiple locations increases the risk of it being compromised, and once it is compromised the attacker can generate valid tokens and impersonate any user in the system.

So for these types of applications, it would be better to have the signing key safely stored in the authentication service, and only used to generate keys, while all other services can verify those tokens without actually having access to the key. And this can actually be accomplished with public-key cryptography.

Public-key cryptography is based on encryption keys that have two components: a public key and a private key. As it name imples, the public key component can be shared freely. There are two workflows that can be accomplished with public-key cryptography:

• Message encryption: If I want to send an encrypted message to someone, I can use that person's public key to encrypt it. The encrypted message can only be decrypted with the person's private key.
• Message signing: If I want to sign a message to certify that it came from me, I can generate a signature with my own private key. Anybody interested in verifying the message can use my public key to confirm that the signature is valid.

There are signing algorithms for JWTs that implement the second scenario above. Tokens are signed with the server's private key, and then they can be verified by anyone using the server's public key, which is freely available to anyone who wants to have it. For the examples that follow I'm going to use the RS256 signing algorithm, which is short for RSA-SHA256.

The pyjwt package does not directly implement the cryptographic signing functions for the more advanced public-key signing algorithms, and instead depends on the cryptography package to provide those. So to use public-key signatures, this package needs to be installed:

(venv) $ pip install cryptography

The next step is to generate a public/private key set (usually called a "key pair") for the application to use. There are a few different ways to generate RSA keys, but one that I like is to use the ssh-keygen tool from openssh:

(venv) $ ssh-keygen -t rsa -b 4096 -m pem
Generating public/private rsa key pair.
Enter file in which to save the key (/home/miguel/.ssh/id_rsa): jwt-key
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in jwt-key.
Your public key has been saved in jwt-key.pub.
The key fingerprint is:
SHA256:ZER3ddV4/smE0rnoNesS+IwCNSbwu5SThfiWWtLYRVM miguel@MS90J8G8WL
The key's randomart image is:
+---[RSA 4096]----+
|       .+E. ....=|
|   .   + . .  ..o|
|    + o +   . oo |
|   . + O   . + ..|
|    = @ S . o + o|
|   o #   . o + o.|
|    * +   = o o  |
|   . . . . = .   |
|        .   o.   |
+----[SHA256]-----+

The -t option to the ssh-keygen command defines that I'm requesting an RSA key pair, and the -b option specifies a key size of 4096 bits, which is considered a very secure key length. The -m option specifies that the key should be generated in PEM format. When you run the command you will be prompted to provide a filename for the key pair, and for this I used jwt-key without any path, so that the key is written to the current directory. Then you will be prompted to enter a passphrase to protect the key, which needs to be left empty.

When the command completes, you are left with two files in the current directory, jwt-key and jwt-key.pub. The former is the private key, which will be used to generate token signature, so you should protect this very well. In particular, you should not commit your private key to your source control, and instead should install on your server directly (you should keep a well protected backup copy of it, in case you ever need to rebuild your server). The .pub file will be used to verify tokens. Since this file has no sensitive information, you can freely add a copy of it on any project that needs to verify tokens.

The process to generate tokens with this key pair is fairly similar to what I showed you earlier. Let's first make a new token:

>>> import jwt
>>> private_key = open('jwt-key').read()
>>> token = jwt.encode({'user_id': 123}, private_key, algorithm='RS256')
>>> token
'eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJ1c2VyX2lkIjoxMjN9.HT1kBSdGFAznrhbs2hB6xjVDilMUmKA-_36n1pLLtFTKHoO1qmRkUcy9bJJwGuyfJ_dbzBMyBwpXMj-EXnKQQmKlXsiItxzLVIfC5qE97V6l6S0LzT9bzixvgolwi-qB9STp0bR_7suiXaON-EzBWFh0PzZi7l5Tg8iS_0_iSCQQlX5MSJW_-bHESTf3dfj5GGbsRBRsi1TRBzvxMUB6GhNsy6rdUhwoTkihk7pljISTYs6BtNoGRW9gVUzfA2es3zwBaynyyMeSocYet6WJri97p0eRnVGtHSWwAmnzZ-CX5-scO9uYmb1fT1EkhhjGhnMejee-kQkMktCTNlPsaUAJyayzdgEvQeo5M9ZrfjEnDjF7ntI03dck1t9Bgy-tV1LKH0FWNLq3dCJJrYdQx--A-I7zW1th0C4wNcDe_d_GaYopbtU-HPRG3Z1SPKFuX1m0uYhk9aySvkec66NBfvV2xEgo8lRZyNxntXkMdeJCEiLF1UhQvvSvmWaWC-0uRulYACn4H-tZiaK7zvpcPkrsfJ7iR_O1bxMPziKpsM4b7c7tmsEcOUZY-IHEI9ibd54_A1O72i08sCWKT5CXyG70MAPqyR0MFlcV7IuDtBW3LCqyvfsDVk4eIj8VcSU1OKQJ1Gl-CTOHEyN-ncV3NslVLaT9Q1C4E7uK2QpS8z0'

The main difference with the previous tokens is that I'm passing the RSA private key as the secret key argument. The value of this key is the entire contents of the jwt-key file. The other difference is that the algorithm requested is RS256 instead of HS256. The resulting token is longer, but otherwise similar to those I generated previously. Like the previous tokens, the payload is not encrypted, so also for these tokens you should never put sensitive information in the payload.

Now that I have the token, I can show you how it can be verified using the public key. If you are trying this with me, exit your Python session and start a new one, to make sure there is no trace of the private key in the Python context. Here is how you can verify the token above:

>>> import jwt
>>> public_key = open('jwt-key.pub').read()
>>> token = 'eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJ1c2VyX2lkIjoxMjN9.HT1kBSdGFAznrhbs2hB6xjVDilMUmKA-_36n1pLLtFTKHoO1qmRkUcy9bJJwGuyfJ_dbzBMyBwpXMj-EXnKQQmKlXsiItxzLVIfC5qE97V6l6S0LzT9bzixvgolwi-qB9STp0bR_7suiXaON-EzBWFh0PzZi7l5Tg8iS_0_iSCQQlX5MSJW_-bHESTf3dfj5GGbsRBRsi1TRBzvxMUB6GhNsy6rdUhwoTkihk7pljISTYs6BtNoGRW9gVUzfA2es3zwBaynyyMeSocYet6WJri97p0eRnVGtHSWwAmnzZ-CX5-scO9uYmb1fT1EkhhjGhnMejee-kQkMktCTNlPsaUAJyayzdgEvQeo5M9ZrfjEnDjF7ntI03dck1t9Bgy-tV1LKH0FWNLq3dCJJrYdQx--A-I7zW1th0C4wNcDe_d_GaYopbtU-HPRG3Z1SPKFuX1m0uYhk9aySvkec66NBfvV2xEgo8lRZyNxntXkMdeJCEiLF1UhQvvSvmWaWC-0uRulYACn4H-tZiaK7zvpcPkrsfJ7iR_O1bxMPziKpsM4b7c7tmsEcOUZY-IHEI9ibd54_A1O72i08sCWKT5CXyG70MAPqyR0MFlcV7IuDtBW3LCqyvfsDVk4eIj8VcSU1OKQJ1Gl-CTOHEyN-ncV3NslVLaT9Q1C4E7uK2QpS8z0'
>>> payload = jwt.decode(token, public_key, algorithms=['RS256'])
>>> payload
{'user_id': 123}

This example looks nearly identical to the previous ones, but the important fact is that we are ensuring this token is valid without access to any sensitive information. The server's public key presents no risk, so it can be freely shared with the world. And in fact, anybody would be able to verify the tokens that your application generates with this key. To prove this point, let me share with you my public key:

>>> public_key = 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCcjWidoIPNRc3IN1hoGeOdSvDkBDK3W3P7/4HxLf62nvUQVczL3FG+dG9KSRnzuvRoUi1o3TASO3Yn72FSfaLPE/JmOtpu/IGuB/oF/CrJqEHA/08n0xkNQK8kwdIqayKPS84PVOm8XomNijMpUCahqu9cGZDPhlgqD8PAxw4e1ZQSizWj0hTSCR78dmHAEr5oXryP6uD0Mw/KGKYel/KTMu00dShWPzHnJeLaYvKgMJKPN6pqhsWFQsNUDnKd9tgn3NSPeHECnnBbUxB2BeuVz72+HnyFWah3mpGH4Dr+9rjRXiPg2AYxgR3U93AEQ6osefxeIKUSCXWx1txNV07QzwFVag4vPBmrA9XktC7i5EP91wxUOsyzhG8geXKuDHmE+/7U3AsExHYFkBLqMnW92CaTeQ408xsRXjxWjSNHpfqhZVxGY5Eh8L3NVqgRg1LdnZYHpovi1iP4Zx2Z7Nb5F9ejuMsA+v/D0WL3c6bhwU8BKdD7YZDG2tpzq6PHt+NarGkcWWh9/p/SIJoZi+e35mjcUMfnRD8w/ouL0sTnxebT7xBCVucfRoMPA67USoChDpc+pNsdtsqlQOZMgpPZYfjIyCThv5mwjEKHnytBq46ULOFlHt0opplDANnDsvWwqEobhACZM+n2ZNtu36eoc3bC/Hak8ACEi5DixirF0w== miguel@MS90J8G8WL'

You can now take this public key and validate the token that I generated, and letting you validate the tokens does not introduce any security risks for me. I'm still the only person in the world that can generate new tokens.

Conclusion

I hope those of you who were using JWTs with the popular HS256 algorithm are now ready to introduce RS256 or any of the other public-key signature options available.

Let me know if you have any questions in the comment area below!