Apple

This is the CPython source code repository - the reference implementation of the Python programming language. It contains the Python interpreter, the complete standard library, build systems for multiple platforms (Windows, macOS, Linux, Android, iOS), and all tooling needed to compile and distribute Python itself.

Key Features

Asyncio Framework
Multiprocessing and Concurrency
Cross-Platform Build System
Import Machinery
Compression Libraries
Database and Serialization
IDLE IDE
AST and Parser Tools

Entry Points

Start here when working with this codebase

Apple Platform Build Entry

Apple/__main__.py

Orchestrates building, packaging, and testing CPython XCframeworks for Apple platforms (iOS, macOS, tvOS, watchOS)

Start here when adding Apple platform support, modifying iOS/macOS build process, or adding new architecture targets

Android Platform Entry

Android/android.py

Handles Android port automation - source configuration, cross-compilation, and device testing

Start here when modifying Android support, adding Android-specific features, or debugging Android test infrastructure

Documentation Build Entry

Doc/conf.py

Central Sphinx configuration for building Python's official documentation

Start here when modifying documentation build process, adding new doc extensions, or changing doc output formats

Interpreter Path Configuration

Modules/getpath.py

Runtime path configuration initialization - determines sys.path, prefix, and exec_prefix at interpreter startup

Start here when modifying how Python finds modules, changing installation paths, or debugging import path issues

AST Grammar Definition

Parser/asdl.py

Parses ASDL specification files into structured objects for AST code generation

Start here when modifying Python's abstract syntax tree structure or adding new AST node types

Future Features Registry

Lib/__future__.py

Registry of Python language features with optional/mandatory release versions and compiler flags

Start here when adding new __future__ imports or deprecating language features

Collections ABC Hierarchy

Lib/_collections_abc.py

Defines Abstract Base Class hierarchy for all Python collection types

Start here when adding new collection ABCs, modifying type checking behavior, or extending collection interfaces

How to Make Changes

Common modification patterns for this codebase

Add a new AST node type

Parser/Python.asdl

Add the new node definition to the ASDL grammar file

Pattern: Follow existing node definitions like 'expr = BoolOp(boolop op, expr* values)' - specify constructor name, fields with types

Parser/asdl.py

Verify ASDL parser handles your new construct (usually no changes needed)

Pattern: Check ASDLParser class methods like p_product and p_sum for grammar rules

Parser/asdl_c.py

Regenerate C code by running this script - it auto-generates from ASDL

Pattern: Run 'python Parser/asdl_c.py Parser/Python.asdl' to regenerate Include/internal/pycore_ast.h and Python/Python-ast.c

Lib/_ast_unparse.py

Add unparse method for the new node to convert AST back to source

Pattern: Follow visit_BoolOp or visit_Compare methods - create visit_YourNode that writes proper Python syntax

Lib/ast.py

Add any high-level helper functions for the new node type

Pattern: Follow existing helpers like literal_eval or walk

Conventions to follow:

Node names use CamelCase (e.g., BoolOp, FunctionDef)Field names use snake_case (e.g., func_name, body)Optional fields marked with '?' suffixSequence fields marked with '*' suffixAlways add corresponding test cases in Lib/test/test_ast.py

Add a new standard library module

Lib/yourmodule.py

Create the new module file in Lib/ directory

Pattern: Follow Lib/pathlib/__init__.py for class-based modules or Lib/_aix_support.py for utility modules - include __all__ export list and proper docstrings

Lib/test/test_yourmodule.py

Create comprehensive test file

Pattern: Follow Lib/test/test_pathlib/test_pathlib_abc.py structure - use unittest.TestCase, group related tests in classes

Doc/library/yourmodule.rst

Create documentation file

Pattern: Follow Doc/library/pathlib.rst format - include module docstring, class/function documentation with examples

Doc/library/index.rst

Add module to documentation index under appropriate category

Pattern: Add 'yourmodule' to the appropriate toctree section

Lib/__future__.py

Only if module introduces new language features requiring __future__ import

Pattern: Follow existing _Feature definitions with mandatory_release and compiler_flag

Conventions to follow:

Module names use lowercase with underscores (e.g., my_module)Private modules start with underscore (e.g., _internal_helper)Define __all__ to control 'from module import *' behaviorInclude module-level docstring explaining purposeAdd NEWS entry in Misc/NEWS.d/next/Library/ using blurb tool

Add a new collection Abstract Base Class

Lib/_collections_abc.py

Define new ABC class with @abstractmethod decorators

Pattern: Follow Mapping class definition - inherit from appropriate parent ABC, define __slots__ = (), use @abstractmethod for required methods, provide default implementations for optional methods

Lib/_collections_abc.py

Add to __all__ list at top of file

Pattern: Add string name to __all__ tuple in alphabetical order

Lib/_collections_abc.py

Pattern: Follow 'Mapping.register(dict)' pattern at bottom of file

Lib/collections/abc.py

Verify re-export (usually automatic via import *)

Pattern: Check that 'from _collections_abc import *' captures your new ABC

Lib/test/test_collections.py

Add test cases for the new ABC

Pattern: Follow TestOneTrickPonyABCs or TestCollectionABCs classes - test isinstance checks, registration, and method behavior

Conventions to follow:

ABC names use CamelCase (e.g., MutableMapping)Use __slots__ = () to prevent instance dictionariesMark required methods with @abstractmethodProvide mixin methods with default implementations where sensibleRegister all built-in types that satisfy the interface

Add a new asyncio feature

Lib/asyncio/base_events.py

Add core event loop method if feature requires loop integration

Pattern: Follow create_task or run_in_executor methods - define in BaseEventLoop class with proper async/await syntax

Lib/asyncio/tasks.py

Add task-related functionality

Pattern: Follow Task class or gather/wait functions - use @coroutine or async def appropriately

Lib/asyncio/streams.py

Add stream-related functionality if applicable

Pattern: Follow StreamReader/StreamWriter classes for I/O operations

Lib/asyncio/__init__.py

Export new public APIs in __all__

Pattern: Add to __all__ list and import from appropriate submodule

Lib/test/test_asyncio/test_tasks.py

Add comprehensive async tests

Pattern: Follow existing test classes - use self.loop.run_until_complete() or asyncio.run() for test execution

Conventions to follow:

Use async def for coroutines, not @asyncio.coroutine decoratorRaise proper exceptions from asyncio.exceptions moduleSupport cancellation via CancelledErrorDocument timeout behavior explicitlyTest with multiple event loop implementations

Add a new compression algorithm

Lib/compression/yourcodec/__init__.py

Create new compression module package

Pattern: Follow Lib/compression/zstd/__init__.py structure - define compress(), decompress(), open() functions and Compressor/Decompressor classes

Lib/compression/yourcodec/_yourcodecmodule.c

Create C extension module if wrapping native library

Pattern: Follow Modules/_zstd/_zstdmodule.c - use PyModule_Create, define PyMethodDef array

Lib/compression/__init__.py

Pattern: Add import and include in __all__ list

Lib/test/test_yourcodec.py

Create test suite

Pattern: Follow Lib/test/test_zstd.py - test compress/decompress roundtrip, streaming, error handling

PCbuild/get_external.py

Add external dependency download for Windows builds if needed

Pattern: Follow existing get_external patterns for zlib, openssl

Conventions to follow:

Provide both one-shot (compress/decompress) and streaming (Compressor/Decompressor) APIsSupport file-like interface via open() functionUse consistent parameter names: level, threads, etc.Handle partial reads/writes in streaming modeDocument compression level ranges and defaults

Add a new XML-RPC server feature

Lib/xmlrpc/server.py

Extend SimpleXMLRPCServer or SimpleXMLRPCRequestHandler class

Pattern: Follow DocXMLRPCServer class extension pattern - override do_GET for new HTTP endpoints, override _dispatch for new RPC behavior

Lib/xmlrpc/server.py

Add introspection methods if extending RPC capabilities

Pattern: Follow system.listMethods, system.methodHelp pattern in SimpleXMLRPCDispatcher

Lib/xmlrpc/client.py

Add client-side support if needed

Pattern: Follow ServerProxy class for client implementation

Lib/test/test_xmlrpc.py

Add server and client tests

Pattern: Follow SimpleServerTestCase class - use threading for server, test both success and error cases

Conventions to follow:

RPC paths default to ('/', '/RPC2') - maintain compatibilityUse register_function() for adding callable methodsSupport both instance and function registrationHandle XML-RPC Fault exceptions properlyDocument security implications of allow_dotted_names

Add Windows build dependency

PCbuild/get_external.py

Add dependency download configuration

Pattern: Follow existing EXTERNALS dict entries - specify URL, hash, and extraction path

PCbuild/prepare_ssl.py

Add preparation script if dependency needs build configuration (like OpenSSL)

Pattern: Follow prepare_ssl.py structure - handle architecture detection, run configure/build commands

PCbuild/python.props

Add MSBuild properties for include/lib paths

Pattern: Follow existing PropertyGroup entries for external dependencies

PCbuild/readme.txt

Document the new dependency and build requirements

Pattern: Follow existing dependency documentation format

Conventions to follow:

Use HTTPS URLs for downloadsInclude SHA256 hash verificationSupport both x86 and x64 architecturesHandle ARM64 if applicableDocument minimum version requirements

Add a new future feature

Lib/__future__.py

Add _Feature instance for the new feature

Pattern: Follow 'annotations = _Feature((3, 7, 0, "beta", 1), (3, 11, 0, "alpha", 0), CO_FUTURE_ANNOTATIONS)' - specify optional_release, mandatory_release, and compiler_flag

Include/compile.h

Define CO_FUTURE_* compiler flag constant

Pattern: Follow existing CO_FUTURE_DIVISION, CO_FUTURE_ANNOTATIONS definitions

Python/compile.c

Handle the compiler flag in compilation

Pattern: Check for flag in compiler_mod and adjust code generation accordingly

Lib/test/test___future__.py

Add tests for the new feature flag

Pattern: Follow existing test patterns - verify feature attributes and compiler behavior

Conventions to follow:

Feature names use lowercase (e.g., annotations, division)optional_release is when 'from __future__ import X' becomes availablemandatory_release is when behavior becomes default (feature import becomes no-op)Use (major, minor, micro, releaselevel, serial) tuples for versionsCompiler flags must be powers of 2 and not conflict with existing flags

Add Android platform support for a module

Android/android.py

Add module to build configuration if it requires special handling

Pattern: Follow existing module handling in configure_build() function

Lib/_android_support.py

Add Android-specific support code if module needs platform adaptation

Pattern: Follow TextIOWrapper redirection pattern for stdout/stderr

Android/testbed/app/src/main/python/

Add Android-specific tests

Pattern: Follow existing test structure in testbed app

Lib/test/test_yourmodule.py

Add skipUnless/skipIf decorators for Android-incompatible tests

Pattern: Use @unittest.skipUnless(hasattr(sys, 'getandroidapilevel'), 'Android only') or similar

Conventions to follow:

Check sys.platform == 'android' for platform detectionUse sys.getandroidapilevel() for API level checksRoute output through Android logging systemHandle missing /dev/tty and terminal featuresTest on multiple Android API levels

Freeze a module into the interpreter

Programs/_freeze_module.py

Understand the freezing process - this script converts .py to frozen C code

Pattern: Run 'python Programs/_freeze_module.py module_name Lib/module.py Python/frozen_module.h' to generate frozen bytecode

Python/frozen.c

Add entry to _PyImport_FrozenModules array

Pattern: Follow existing entries like {"_frozen_importlib", _Py_M__importlib__bootstrap, ...}

Makefile.pre.in

Add freeze target for the module

Pattern: Follow existing FROZEN_FILES targets

Programs/freeze_test_frozenmain.py

Reference for testing frozen module behavior

Pattern: Follow the test pattern for verifying frozen bytecode arrays

Conventions to follow:

Frozen modules load faster but cannot be modified at runtimeUse for bootstrap modules required before import system is readyRegenerate frozen files when source changesTest both frozen and unfrozen versionsDocument why module needs to be frozen

Coding Conventions

Standards and patterns used in this codebase

Documentation

Module docstrings must include comprehensive descriptions with copyright notices, version info, and detailed explanations of module purpose and usage

"""This is the Sphinx configuration file for the Python documentation.

All configuration values have a default value; values that are commented out
serve to show the default value.
"""

Doc/conf.pyLib/__future__.py

Imports

Group imports in order: standard library, third-party, local imports, with blank lines between groups. Use explicit imports over wildcard imports

import dataclasses
import os
import re
import subprocess
import sys
from pathlib import Path
from os.path import relpath

Apple/__main__.pyAndroid/android.pyParser/asdl_c.py

CLI Design

Use argparse with subparsers for complex CLI tools, defining common arguments at parent level and command-specific arguments in subparsers

def parse_args():
    parser = argparse.ArgumentParser()
    subcommands = parser.add_subparsers(dest="subcommand")
    build = subcommands.add_parser("build", help="Build a framework")
    build.add_argument("--target", default="arm64-apple-ios")
    return parser.parse_args()

Apple/__main__.pyAndroid/android.py

Error Handling

Define custom exception classes for domain-specific errors, inheriting from appropriate base exceptions

class ASDLSyntaxError(Exception):
    def __init__(self, lineno, msg=None, filename=None):
        self.lineno = lineno
        self.msg = msg
        self.filename = filename

Parser/asdl.py

Code Generation

Use dedicated visitor pattern classes with emit() methods for generating code, tracking indentation level explicitly

class EmitVisitor(ASDLVisitor):
    def __init__(self, file):
        self.file = file
    def emit(self, s, depth, reflow=True):
        if reflow:
            lines = reflow_c_string(s, depth)
        else:
            lines = [s]
        for line in lines:
            if line:
                self.file.write(" " * TABSIZE * depth + line + "\n")

Parser/asdl_c.py

Constants

Define module-level constants using UPPER_CASE naming, grouped at top of file after imports

SRCDIR = Path(__file__).resolve().parent.parent
TABSIZE = 4
MAX_COL = 80
DEFAULT_HOST = "$(CC) --print-multiarch"

Parser/asdl_c.pyApple/__main__.pyAndroid/android.py

Type Hints

Use dataclasses with type annotations for structured data, especially for AST nodes and configuration objects

@dataclasses.dataclass
class Module:
    name: str
    dfns: list

@dataclasses.dataclass
class Type:
    name: str
    value: object

Parser/asdl.py

Subprocess Execution

Create wrapper functions for subprocess calls that handle logging, error checking, and environment configuration consistently

def run(*args, check=True, quiet=False, **kwargs):
    if not quiet:
        log(" ".join(map(str, args[0])))
    return subprocess.run(*args, check=check, **kwargs)

Apple/__main__.pyAndroid/android.pyPCbuild/get_external.py

Path Handling

Use pathlib.Path for all filesystem operations, resolving paths relative to script location using __file__

SRCDIR = Path(__file__).resolve().parent.parent
BUILD_DIR = SRCDIR / "build"
CROSS_BUILD_DIR = BUILD_DIR / "cross-build"

Apple/__main__.pyAndroid/android.pyPrograms/_freeze_module.py

Main Entry Point

Use if __name__ == '__main__' guard with sys.exit() wrapping main function for proper exit code handling

if __name__ == "__main__":
    sys.exit(main())

Apple/__main__.pyAndroid/android.pyPCbuild/get_external.py

Feature Flags

Use named tuples or dataclasses for feature flag definitions with mandatory and optional release version fields

_Feature = collections.namedtuple('_Feature', [
    'optional_release',
    'mandatory_release',
    'compiler_flag'
])

annotations = _Feature((3, 7, 0, 'beta', 1), (3, 14, 0, 'alpha', 1), CO_FUTURE_ANNOTATIONS)

Lib/__future__.py

Logging

Implement simple logging functions that respect verbosity flags and write to stderr for build/automation scripts

def log(msg=""):
    print(msg, file=sys.stderr, flush=True)

def log_cmd(args):
    if verbose:
        log(" ".join(map(str, args)))

Apple/__main__.pyAndroid/android.py

Apple

Key Features

Entry Points

How to Make Changes

Add a new AST node type

Add a new standard library module

Add a new collection Abstract Base Class

Add a new asyncio feature

Add a new compression algorithm

Add a new XML-RPC server feature

Add Windows build dependency

Add a new __future__ feature

Add Android platform support for a module

Freeze a module into the interpreter

Coding Conventions

Add a new future feature