Python¶
Dataclasses¶
https://stackoverflow.com/questions/55307017/pickle-a-frozen-dataclass-that-has-slots
https://rednafi.com/python/statically_enforcing_frozen_dataclasses/
For dataclasses we can set frozen=True, slots=True:
from dataclasses import dataclass
@dataclass(frozen=True, slots=True)
class Frozen:
x: int
y: int
l: list[int] # <-- still mutable, use tuples instead
Set defaults:
from dataclasses import dataclass, field
@dataclass
class A:
x: int = 0
y: int = field(default=0) # same as x
Validation approaches:
Use
__post_init__- Adding all logic to__init__: example - Definingvalidate_*functions and calling them in a loop in__post_init__: src - Instead ofvalidate_*functions you can use metadata, like thisage: int = field(metadata={"validate": lambda value: value > 0})- Type checking is somewhat tricky (here is a simpler non-recursive type-checking version), use 3rd party libraries insteadFor mutable dataclasses also define
__setattr__Use 3rd party libs like
pydantic,msgspec
Decorators¶
Primer on Python Decorators - multiple examples, but no info on typing decorators
Typing a decorator:
from functools import wraps
from typing import Callable, TypeVar, ParamSpec
from typing import reveal_type
ParamsTs = ParamSpec('ParamsTs')
ReturnT = TypeVar('ReturnT')
def debug(
fun: Callable[ParamsTs, ReturnT],
) -> Callable[ParamsTs, ReturnT]:
@wraps(fun)
def inner(
*args: ParamsTs.args,
**kwargs: ParamsTs.kwargs,
) -> ReturnT:
...
return fun(*args, **kwargs)
return inner
@debug
def concat(a: str, b: str) -> str:
"""Concatenate two strings."""
return a + b
reveal_type(concat)
# Revealed type is "def (a: str, b: str) -> str"
print(concat.__doc__)
# Will print: "Concatenate two strings."
Decorator having more or less arguments than decorated function:
from functools import wraps
from typing import Callable, Concatenate, ParamSpec, TypeVar
ParamsTs = ParamSpec('ParamsTs')
ReturnT = TypeVar('ReturnT')
class User: ...
class Request: ...
class Response: ...
def with_user(
fun: Callable[Concatenate[User, ParamsTs], ReturnT],
) -> Callable[P, ReturnT]:
@wraps(fun)
def inner(*args: ParamsTs.args, **kwargs: ParamsTs.kwargs) -> ReturnT:
user = User()
return fun(user, *args, **kwargs)
return inner
@with_user
def handle_request(
user: User,
request: Request,
) -> Response:
...
request = Request()
response = handle_request(request)
Immutability¶
Immutable structures¶
Immutable versions of standard types:
Immutable
lististuple.Immutable
setisfrozenset.Immutable
bytearrayisbytes.dictdoesn’t have an immutable version but it hastypes.MappingProxyTypewrapper that makes it immutable
from types import MappingProxyType
orig = {1: 2}
immut = MappingProxyType(orig)
immut[3] = 4
# TypeError: 'mappingproxy' object does not support item assignment
orig[3] = 4
immut[3]
# 4
For dataclasses we can set frozen=True, slots=True.
Refer to Dataclasses section for details and gotchas.
3rd party libraries:
Final type¶
MAX_SIZE: Final = 9000
MAX_SIZE += 1 # Error reported by type checker
Parsing¶
Faster, more memory-efficient Python JSON parsing with msgspec - For large files use ijson - Msgspec is used by litestar web framework (src: HN)
Dataclasses vs Attrs vs Pydantic¶
Pattern Matching¶
https://mathspp.com/blog/pydonts/structural-pattern-matching-tutorial
https://mathspp.com/blog/pydonts/structural-pattern-matching-anti-patterns
https://earthly.dev/blog/structural-pattern-matching-python/
https://www.infoworld.com/article/3609208/how-to-use-structural-pattern-matching-in-python.html
Memoization and Caching¶
String Formatting¶
# Print key=value
print(f"{key=}")
# Floats
pi = 3.141592
precision = 3
print(f"{pi:.{precision}f}") # "3.142"
# Percentage
val = 1.255
print(f"{val:%}") # 125.500000%
print(f"{val:.0%}") # 125%
print(f"{val:.1%}") # 125.5%
# Scientific notation
val = 1.2345e3
print(f"{val:e}") # 1.234500e+03
print(f"{val:E}") # 1.234500E+03
print(f"{val:.2e}") # 1.23e+03
# Integers
val = 1_000_000
print(f"{val:,d}") # 1,000,000
print(f"{val:_d}") # 1_000_000
# Numbers
val = 1
print(f"{val =: n}") # val = 1
# Dates
day = date(year=2022, month=9, day=1)
print(f"{day}")
print(f"{day:%Y-%m-%d}") # default appearance
print(f"{day:%Y/%m/%d}") # use `/` as separator
print(f"{day:%Y %b %d}") # 2022 Sep 01
print(f"{day:%Y %B %d}") # 2022 September 01
print(f"{day:%Y %b %d (%A)}") # 2022 Sep 01 (Thursday)
# Datetime
day_and_time = datetime(year=2022, month=9, day=1, hour=17, minute=30, second=45)
now = datetime.now()
print(f"{day_and_time}") # 2022-09-01 17:30:45
print(f"{now}") # 2022-12-08 15:49:37.810347
print(f"{now:%Y-%m-%d %H:%M:%S.%f}") # 2022-12-08 15:49:37.810347
print(f"{now:%Y-%m-%d %H:%M:%S.%f}"[:22]) # 2022-12-08 15:49:37.81
# Padding
val = 1
print(f"{val:3d}") # " 1"
print(f"{val:03d}") # "001"
# Positive/negative sign
positive = 1.23
negative = -1.23
print(f"1: {positive:+.2f} {negative:+.2f}") # 1: +1.23 -1.23
print(f"2: {positive:-.2f} {negative:-.2f}") # 2: 1.23 -1.23
print(f"3: {positive: .2f} {negative: .2f}") # 3: 1.23 -1.23
Profiling¶
- plasma-umass/scalene
has comparison with other profilers: img
- bloomberg/memray
has pytest plugin
- joerick/pyinstrument
can integrate with FastAPI, call stack is available when
?profile=trueis added to the url