Python JSON Tutorial: Parse, Read, Write, and Manipulate JSON Data

The Basics: json.loads() and json.dumps()

Python's json module has four main functions. Two work with strings, two work with files:

Parsing JSON strings with json.loads()

import json

# Parse a JSON string into a Python dictionary
json_string = '{"name": "Alice", "age": 30, "active": true}'
data = json.loads(json_string)

print(data["name"])    # Alice
print(data["age"])     # 30
print(data["active"])  # True (Python bool, not JSON true)
print(type(data))      # <class 'dict'>

Converting Python to JSON with json.dumps()

import json

user = {
    "name": "Alice",
    "age": 30,
    "skills": ["Python", "SQL", "Docker"],
    "active": True,
    "manager": None
}

# Convert to JSON string
json_string = json.dumps(user)
print(json_string)
# {"name": "Alice", "age": 30, "skills": ["Python", "SQL", "Docker"], "active": true, "manager": null}

# Pretty print with indentation
pretty = json.dumps(user, indent=2)
print(pretty)

Python to JSON Type Mapping

Python and JSON types don't map 1:1. Understanding the conversion rules prevents surprises:

Types not in this list — datetime, set, bytes, custom classes — will raise TypeError: Object of type X is not JSON serializable. We'll cover how to handle these below.

Reading and Writing JSON Files

Reading a JSON file

import json

# Read and parse a JSON file
with open("config.json", encoding="utf-8") as f:
    config = json.load(f)

print(config["database"]["host"])  # localhost

Writing a JSON file

import json

data = {
    "database": {
        "host": "localhost",
        "port": 5432,
        "name": "myapp"
    },
    "debug": False
}

# Write formatted JSON to a file
with open("config.json", "w", encoding="utf-8") as f:
    json.dump(data, f, indent=2, ensure_ascii=False)

# ensure_ascii=False preserves unicode characters like é, ñ, 日本語
# instead of escaping them to \uXXXX

Working with API Responses

The most common use of JSON in Python is parsing API responses. The requests library has built-in JSON support:

import requests

response = requests.get("https://api.github.com/users/octocat")

# Method 1: Use the built-in .json() method
data = response.json()
print(data["login"])  # octocat

# Method 2: Parse manually (useful for error handling)
import json
try:
    data = json.loads(response.text)
except json.JSONDecodeError as e:
    print(f"Invalid JSON: {e}")

# Always check the status code first
if response.status_code == 200:
    data = response.json()
else:
    print(f"API error: {response.status_code}")

Sending JSON in POST requests

import requests

payload = {
    "title": "New Post",
    "body": "Post content here",
    "userId": 1
}

# requests automatically serializes the dict and sets Content-Type
response = requests.post(
    "https://jsonplaceholder.typicode.com/posts",
    json=payload  # use json= not data=
)

print(response.status_code)  # 201
print(response.json())       # response with created resource

Navigating Nested JSON

Real-world JSON is usually nested several levels deep. Here's how to work with it safely:

import json

data = {
    "users": [
        {
            "id": 1,
            "name": "Alice",
            "address": {
                "city": "New York",
                "zip": "10001"
            }
        },
        {
            "id": 2,
            "name": "Bob",
            "address": {
                "city": "San Francisco",
                "zip": "94102"
            }
        }
    ]
}

# Access nested values
city = data["users"][0]["address"]["city"]  # "New York"

# Loop through a list of objects
for user in data["users"]:
    print(f"{user['name']} lives in {user['address']['city']}")

# Safe access with .get() to avoid KeyError
email = data["users"][0].get("email", "not provided")
# Returns "not provided" instead of raising KeyError

Safely accessing deeply nested data

# Helper function for deep access
def deep_get(obj, *keys, default=None):
    """Safely navigate nested dicts/lists."""
    for key in keys:
        try:
            obj = obj[key]
        except (KeyError, IndexError, TypeError):
            return default
    return obj

# Usage
city = deep_get(data, "users", 0, "address", "city")
# Returns None instead of crashing if any key is missing

missing = deep_get(data, "users", 5, "address", "city", default="unknown")
# Returns "unknown"

Handling Non-Serializable Types

Python objects like datetime, set, Decimal, and custom classes can't be serialized to JSON directly. You need a custom encoder:

Using the default parameter

import json
from datetime import datetime, date
from decimal import Decimal

def json_serializer(obj):
    """Handle types that json.dumps() can't serialize."""
    if isinstance(obj, (datetime, date)):
        return obj.isoformat()
    if isinstance(obj, Decimal):
        return float(obj)
    if isinstance(obj, set):
        return list(obj)
    if isinstance(obj, bytes):
        return obj.decode("utf-8")
    raise TypeError(f"Type {type(obj)} is not JSON serializable")

data = {
    "created": datetime.now(),
    "price": Decimal("19.99"),
    "tags": {"python", "json", "tutorial"},
}

result = json.dumps(data, default=json_serializer, indent=2)
print(result)

Using a custom JSONEncoder class

import json
from datetime import datetime

class CustomEncoder(json.JSONEncoder):
    def default(self, obj):
        if isinstance(obj, datetime):
            return obj.isoformat()
        return super().default(obj)

data = {"event": "deploy", "timestamp": datetime.now()}
json.dumps(data, cls=CustomEncoder)

Sorting Keys and Formatting Options

import json

data = {"banana": 2, "apple": 5, "cherry": 1}

# Sort keys alphabetically (useful for consistent diffs)
print(json.dumps(data, sort_keys=True))
# {"apple": 5, "banana": 2, "cherry": 1}

# Custom separators for compact output
print(json.dumps(data, separators=(",", ":")))
# {"banana":2,"apple":5,"cherry":1}  — no spaces, minimal size

# Combine options
print(json.dumps(data, indent=2, sort_keys=True, ensure_ascii=False))

Error Handling

JSON parsing can fail in several ways. Here's how to handle each:

import json

def parse_json_safely(text, source="unknown"):
    """Parse JSON with comprehensive error handling."""
    if not isinstance(text, str):
        print(f"[{source}] Expected string, got {type(text).__name__}")
        return None

    if not text.strip():
        print(f"[{source}] Empty input")
        return None

    try:
        return json.loads(text)
    except json.JSONDecodeError as e:
        print(f"[{source}] JSON error at line {e.lineno}, col {e.colno}: {e.msg}")
        print(f"  Near: ...{text[max(0, e.pos-20):e.pos+20]}...")
        return None

# Usage
data = parse_json_safely('{"name": "Alice",}', source="config.json")
# [config.json] JSON error at line 1, col 18: Expecting property name enclosed in double quotes

Working with Large JSON Files

For JSON files that are too large to fit in memory, use ijson for streaming:

# pip install ijson
import ijson

# Stream-parse a large JSON file without loading it all into memory
with open("large_data.json", "rb") as f:
    # Extract all items from the "users" array one at a time
    for user in ijson.items(f, "users.item"):
        print(user["name"])
        # Each user is parsed individually — memory stays low

Reading JSON Lines (JSONL) format

import json

# JSONL: one JSON object per line — common in logging and data pipelines
with open("events.jsonl", encoding="utf-8") as f:
    for line in f:
        if line.strip():  # skip empty lines
            event = json.loads(line)
            print(event["type"], event["timestamp"])

Python JSON with Dataclasses and Pydantic

Dataclasses

import json
from dataclasses import dataclass, asdict

@dataclass
class User:
    name: str
    age: int
    email: str

# Create from JSON
data = json.loads('{"name": "Alice", "age": 30, "email": "alice@example.com"}')
user = User(**data)
print(user.name)  # Alice

# Convert back to JSON
json_string = json.dumps(asdict(user), indent=2)

Pydantic (recommended for production)

from pydantic import BaseModel, EmailStr

class User(BaseModel):
    name: str
    age: int
    email: EmailStr

# Parse and validate in one step
user = User.model_validate_json('{"name": "Alice", "age": 30, "email": "alice@example.com"}')
print(user.name)  # Alice

# Invalid data raises a clear error
try:
    User.model_validate_json('{"name": "Alice", "age": "thirty", "email": "not-email"}')
except Exception as e:
    print(e)  # Detailed validation errors

# Serialize back to JSON
print(user.model_dump_json(indent=2))

Common Mistakes and How to Avoid Them

• Using eval() instead of json.loads() — never do this. eval() executes arbitrary code and is a critical security vulnerability.
• Forgetting encoding="utf-8" — on Windows, open() defaults to the system encoding (often cp1252), which corrupts non-ASCII characters. Always specify UTF-8.
• Using data= instead of json= in requests — requests.post(url, data=dict) sends form data, not JSON. Use json=dict to send JSON.
• Modifying a dict while iterating — if you're filtering JSON data, build a new dict instead of deleting keys during iteration.
• Assuming key order — Python dicts preserve insertion order (3.7+), but JSON spec says order is not significant. Don't rely on it from external sources.
• Not handling None vs missing keys — {"age": null} and {} are different. Use .get("age") with a default to handle both.

Quick Reference: json Module Cheat Sheet

import json

# String ↔ Python
data = json.loads('{"key": "value"}')       # parse string
text = json.dumps(data)                      # serialize to string
text = json.dumps(data, indent=2)            # pretty print
text = json.dumps(data, sort_keys=True)      # sorted keys

# File ↔ Python
with open("f.json") as f: data = json.load(f)         # read file
with open("f.json", "w") as f: json.dump(data, f)     # write file

# Options
json.dumps(data, ensure_ascii=False)         # preserve unicode
json.dumps(data, separators=(",", ":"))      # compact output
json.dumps(data, default=str)                # serialize anything (fallback to str)