Handling JSON with Python

JSON is the go-to syntax for storing and exchanging data over the internet. Almost all programming languages support JSON to transfer information across the web. In Python, you convert JSON to a Python dictionary.

A weather app can use a RESTful API to pull JSON from weather data centers to show your current temperatures. Isn’t that awesome?! The applications are incredible, from dad jokes sent to your phone every day, that’s my favorite, to today’s stock prices, or maybe you want to keep track of Bitcoin.

Now, doesn’t that make you want to learn how to put JSON to work for you? Your Journey Into Python begins now!

This topic is part of our Introduction to Python Course. (It’s Free). We like to keep things simple and leverage Google’s Colab which allows you to write python right in your browser. You can follow along with our notebook if you would like. If you want more details on this topic you can visit Python.org.

What is JSON?

JavaScript Object Notation (JSON) is a text format used to represent structured data, like a Python dictionary, based on JavaScript object syntax {[""]}. It is commonly used for transmitting data in web applications to display the collected information.

JSON is built on two structures, a collection of name/value pairs and an ordered list of values. This is realized as an array, vector, list, dictionary, or sequence in most languages.

This is how JSON is setup:

An object is an unordered set of name/value pairs {}. An array is an ordered collection of values []. Values can be a string in double quotes, a number, true, false, null, object, or an array. These structures can be nested.

Strings are a sequence of blank or more characters wrapped in double-quotes. And Numbers, except that the octal and hexadecimal formats are not used. Here, you can visualize how JSON can be set up JSON.org. Here’s an example of a JSON object.

theObject = `{
  "string":"string",
  "number":30,
  "true":True,
  "false":False,
  "null":None,
  "tuple":("tuple1","tuple2"),
  "list":["list1","list2"],
  "array":[
    {"string":"string","number":10},
    {"true":True,"nestedList":["list1","list2"]},
    {"ect...":"Can get as crazy as you need!!!"},
  ]
}`

Converting from JSON to a Python Dictionary

Python has a built-in JSON package, which can be imported to work with JSON data. If you have a JSON string, you can parse it by using the json.loads() method. Take notice of the single quotes on names variable; that’s the difference between a JSON and Python dictionary.

import json

names = '{"names":{"name1":"John","name2":"Jack","name3":"Jane"}}'

jsonPython = json.loads(names)

print(jsonPython["names"])

{'name1': 'John', 'name2': 'Jack', 'name3': 'Jane'}

Convert from a Python Dictionary to JSON

If you have a Python object, you can convert it into a JSON string by using the json.dumps() method.

Python	JSON	Python	JSON
dict	Object	int	Number
list	Array	float	Number
tuple	Array	True	true
str	String	False	false
None	null

import json

pythonDict = {
  "badJoke":
 {
    "question1":"Why did the slug call the police?",
    "answer1":"Because he was a'salted'."
 },
  "goodJoke":
 {
    "question2":"Why did the child get arrested for refusing to take a nap?",
    "answer":"Because she was resisting a'rest'."
 }
}

pythonJson = json.dumps(pythonDict, indent=4)

# When converted becomes one long string
print(pythonJson)

{
    "badJoke": {
        "question1": "Why did the slug call the police?",
        "answer1": "Because he was a'salted'."
    },
    "goodJoke": {
        "question2": "Why did the child get arrested for refusing to take a nap?",
        "answer": "Because she was resisting a'rest'."
    }
}

Formatting the printed Result

JSON can be pretty ugly when printed with the standard python print function. The best method is to copy the JSON to an interface that can read it. For example, you could copy the output and paste it into a cell. This will help you with the structure because you can fold the JSON to help you identify what the JSON structure is. You could also do this in Visual Studio Code or almost any other IDE.

# Hover the mouse near the line number to see the code folding 

theObject = { # Notice you can fold here
  "string":"string",
  "number":30,
  "true":True,
  "false":False,
  "null":None,
  "tuple":("tuple1","tuple2"),
  "list":["list1","list2"],
  "array":[ # or you can fold here
    {"string":"string","number":10},
    {"true":True,"nestedList":["list1","list2"]},
    {"ect...":"Can get as crazy as you need!!!"},
  ]
}

However, if you are in a pinch, you can also use the pretty print function pprint(). It is easy to use but has numerous features. You can also use indent= with the json.dumps() method. To read more about print, check it out here: Python.org

from pprint import pprint

pprint(theObject)

{'array': [{'number': 10, 'string': 'string'},
           {'nestedList': ['list1', 'list2'], 'true': True},
           {'ect...': 'Can get as crazy as you need!!!'}],
 'false': False,
 'list': ['list1', 'list2'],
 'null': None,
 'number': 30,
 'string': 'string',
 'true': True,
 'tuple': ('tuple1', 'tuple2')}

# Notice how much better pprint looks
print(theObject)

{'string': 'string', 'number': 30, 'true': True, 'false': False, 'null': None, 'tuple': ('tuple1', 'tuple2'), 'list': ['list1', 'list2'], 'array': [{'string': 'string', 'number': 10}, {'true': True, 'nestedList': ['list1', 'list2']}, {'ect...': 'Can get as crazy as you need!!!'}]}

Iterate Through Nested JSON Object to Get a Specific Value

Let’s pull a few things from the JSON object “theObject”. First, let’s start with the second list value, “list2”. Now let’s start by determining what keys are available using the keys function.

theObject.keys()

dict_keys(['string', 'number', 'true', 'false', 'null', 'tuple', 'list', 'array'])

Note here that the list is one of the available keys. Let’s pull that out.

theObject['list']

['list1', 'list2']

First, we get our list. Next, let’s pull out the second index.

theObject['list'][1]

'list2'

Second, we can try to grab nested values deeper in theObject. At this time, we will try and grab the second index of the nestedList.

theObject['array']

[{'number': 10, 'string': 'string'},
 {'nestedList': ['list1', 'list2'], 'true': True},
 {'ect...': 'Can get as crazy as you need!!!'}]

Now we see that the list contains the nestedList, and it is the second index.

theObject['array'][1]

{'nestedList': ['list1', 'list2'], 'true': True}

So now the data is a dictionary. We will have to pull out the nestedList as a key.

theObject['array'][1]['nestedList']

['list1', 'list2']

Ok, now we have the list. We want the second value, which is index 1 in the list.

theObject['array'][1]['nestedList'][1]

'list2'

Phew, we got it.

JSON can, at times, be quite complex, so knowing your data types is essential. Remember, when in doubt, you can use type() on the object to determine its data type. You might be thinking, does this happen in the real world? It does. I was working on automating file analysis and was leveraging the VirusTotal API. Just look how complex this API object is here at VirusTotal.com.

Order the Result

The json.dumps() method has parameters to order the keys in the result if you need them in alphabetical order. Use the sort_keys parameter to specify if the result should be sorted or not.

json.dumps(theObject, sort_keys=True)
pprint(pythonDict)

{'badJoke': {'answer1': "Because he was a'salted'.",
             'question1': 'Why did the slug call the police?'},
 'goodJoke': {'answer': "Because she was resisting a'rest'.",
              'question2': 'Why did the child get arrested for refusing to '
                           'take a nap?'}}

Cheat Sheet Python JSON

Friendly link to Cheat Sheet Page

Python JSON

JSON is a syntax for storing and exchanging data that uses text, written with JavaScript object notation.

Syntax

object = {
  "key":"value",
  "key":"value"
}

Helpful functions

import json
json.loads(json)
json.dumps(dict, indent=4)
pprint(json)
x.keys()

Common mistakes

Syntax: missing or wrong location of commas, colons, double-quotes.

Trying to find values before converting JSON to Python dictionary. Use the type() function if you’re not sure what your variable is.

Typical Use

Most web applications use JSON to transmit their data.