Главная » Error » pedantic-python-decorators from LostInDarkMath – Coder Social

pedantic-python-decorators from LostInDarkMath – Coder Social

pedantic-python-decorators’s Introduction

Data serialization caveats

Before I conclude, I’d like to talk about two caveats I faced while using these libraries.

Variable naming

When you receive raw, key-value-like data, e.g. in JSON form, the keys may follow all kinds of naming conventions. For instance, you may receive {“user-name”: “Peter”} (kebab case), {“userName”: “Peter”} (camel case), or {“user_name”: “Peter”} (snake case). In Python, however, snake case is common, and using any other convention would be extremely weird.

Fortunately, both marshmallow and Pydantic offer support to actually rename fields dynamically. See here for marshmallow and here for Pydantic. In case of Pydantic, the linked docs just cover deserialization – for serialization you simply need to call o.dict(by_alias=True) instead of o.dict().

Data ambiguity and polymorphism

Data processing with both libraries works as expected, as long as the set of classes are deterministic. In particular, deserialization requires prior knowledge (provided by your code) about which schema to use to deserialize a nested dict object you just obtained from somewhere. Let’s take a look at an example which is not deterministic, but ambiguous, because it involves inheritance:

@dataclassclassAnimal:
name: str

@dataclassclassTiger(Animal):
weight_lbs: int

@dataclassclassElephant(Animal):
trunk_length_cm: int

@dataclassclassZoo:
animals: List[Animal]

animals = [Elephant(name=”Eldor”, trunk_length_cm=176), Tiger(name=”Roy”, weight_lbs=405)] zoo = Zoo(animals=animals)Code language:Python(python)

Animal is inherited by Tiger and Elephant. However, serialization considers Zoo.animals to be just Animal objects. Consequently, serializing zoo would result in something like {‘animals’: [{‘name’: ‘Eldor’}, {‘name’: ‘Roy’}]}. The information added by the sub-classes is missing.

Neither Pydantic nor Marshmallow support such kind of data. For marshmallow, there are a few helper projects, but I found them to be tedious to use. A better workaround is to add an attribute to all of your ambiguous classes, e.g. datatype, that contains a string literal for each class. In this case, you also have to reduce ambiguity by explicitly stating a typing.Union field that lists all possibly expected classes, as illustrated below, for marshmallow-dataclass:

from marshmallow.fields import Constant

@dataclassclassTiger(Animal):
weight_lbs: int
datatype: str = field(metadata=dict(marshmallow_field=Constant(“tiger”)), default=”tiger”)

@dataclassclassElephant(Animal):
trunk_length_cm: int
datatype: str = field(metadata=dict(marshmallow_field=Constant(“elephant”)), default=”elephant”)

@dataclassclassZoo:
animals: List[Union[Tiger, Elephant]] # formerly List[Animal]Code language:Python(python)

However, while this approach makes data (de-)serialization of ambiguous data possible, it taints the schema with aspects that are specific to (de-)serialization.

Marshmallow vs. Pydantic – which one is better?

The choice comes down to a matter of personal preferences and needs. Let’s take a look at a few categories:

  • Popularity / stability: it’s a bad idea to choose a library which is not very popular and thus has a high risk of being abandoned. Both marshmallow and Pydantic are about equally popular, with ~5k stars on GitHub each. However, marshmallow development seems to be more on top of things, with ~90 vs. 235 open issues.
  • Performance: the authors of Pydantic built benchmarks which claim that Pydantic’s performance is superior to marshmallow’s. This advantage is only meaningful if you do a lot of data (de)serialization, in which case I recommend you measure the performance impact for one of your more complex schemas. Don’t ever blindly trust benchmarks made by others. In case you work with JSON input/output, you can also use ujson to speed up the conversion between raw str data and nested Python dicts, for both marshmallow and Pydantic.
  • Interoperability with standards: Pydantic comes with built-in support for generating OpenAPI or JSON schema definitions from your model code. There is also an officially endorsed generator tool which converts existing OpenAPI / JSON schema definitions to pydantic model classes. Marshmallow does not offer these capabilities. However, there are third-party projects like marshmallow-jsonschema and apispec which convert marshmallow schema classes to JSON schema or OpenAPI respectively – but not the other way around!

If all you need is serialization and deserialization at specific points in your application, I recommend marshmallow. Pydantic is a good choice if you want type safety throughout the whole lifetime of your objects at run-time, better interoperability with standards, or require very good run-time performance.

Creating the Pydantic model

We can replace the dataclass attribute to be imported from pydantic instead and if we run it with just that change, we will see the validation errors.

from pydantic.dataclasses import dataclass

And this will throw the errors:

pydantic validation errors
pydantic validation errors

Pydantic does support type conversion. So if we passed in the value ‘2’ to an int field, it will be converted and not throw an error.

But data classes have some limitations. And Pydantic provides a BaseModel class which we can extend from. Doing so provides us with features like serialization and first class JSON support. So we will convert our code to:

from pydantic import BaseModel
from typing import Tuple

class Blog(BaseModel):
title: str
author: str
categories: Tuple[str,…]

def main():
blog = Blog(title=None, author=None, categories=None)
print(blog)

main()

The BaseModel implementation is probably the better way to go because of the additional features. It is important to note though that we should not put both the dataclass decorator and the extend from BaseModel since that will not work.

Another thing to note is that BaseModel requires keyword arguments, so while this would have worked with dataclass:

blog=Blog(“Hello World”,”Saransh Kataria”,(“Wisdom”,”Geek”))

With BaseModel, keyword arguments needs to be explicit:

blog=Blog(title=”Hello World”,author=”Saransh Kataria”,categories=(“Wisdom”,”Geek”))

Or we can use **kwargs to do so.

Типы полей

Поддерживает все стандартные типы #python

  • strict types
  • constrained types

Типы:

  • None or type(None) or Literal[None]
  • bool
  • int
  • float
  • str
  • bytes
  • list (принимает list, tuple, set, frozenset, deque и генераторы)
  • tuple (принимает list, tuple, set, frozenset, deque и генераторы)
  • dict
  • set (принимает list, tuple, set, frozenset, deque и генераторы)
  • frozenset (принимает list, tuple, set, frozenset, deque и генераторы)
  • deque (принимает list, tuple, set, frozenset, deque и генераторы)
  • datetime.date
  • datetime.time
  • datetime.datetime
  • datetime.timedelta
  • typing.Any любое значение
  • typing.Annotated анотированное значение
  • typing.TypeVar константа, базирующаяся на этом
  • typing.Union несколько разных типов
  • typing.Optional обертка над Union[x, None]
  • typing.List
  • typing.Tuple
  • subclass of typing.NamedTuple
  • subclass of collections.namedtuple
  • typing.Dict и subclass
  • typing.Set
  • typing.FrozenSet
  • typing.Deque
  • typing.Sequence
  • typing.Iterable зарезевировано под другие итераторы
  • typing.Type
  • typing.Callable
  • typing.Pattern для regex
  • ipaddress.IPv4Address и другие ip…
  • enum.Enum и subclass of enum.Enum
  • enum.IntEnum и subclass
  • decimal.Decimal конвертит в строку, затем в decimal
  • pathlib.Path
  • uuid.UUID
  • ByteSize

Пример с итераторами

fromtypingimport(Deque,Dict,FrozenSet,List,Optional,Sequence,Set,Tuple,Union)frompydanticimportBaseModelclassModel(BaseModel):simple_list:list=Nonelist_of_ints:List[int]=Nonesimple_tuple:tuple=Nonetuple_of_different_types:Tuple[int,float,str,bool]=Nonesimple_dict:dict=Nonedict_str_float:Dict[str,float]=Nonesimple_set:set=Noneset_bytes:Set[bytes]=Nonefrozen_set:FrozenSet[int]=Nonestr_or_bytes:Union[str,bytes]=Nonenone_or_str:Optional[str]=Nonesequence_of_ints:Sequence[int]=Nonecompound:Dict[Union[str,bytes],List[Set[int]]]=Nonedeque:Deque[int]=None

DateTime типы

  • datetime fields can be:
    • datetime, existing datetime object
    • int or float, assumed as Unix time, i.e. seconds (if >= -2e10 or <= 2e10) or milliseconds (if < -2e10or > 2e10) since 1 January 1970
    • str, following formats work:
      • YYYY-MM-DD[T]HH:MM[:SS[.ffffff]][Z or [±]HH[:]MM]]]
      • int or float as a string (assumed as Unix time)
  • date fields can be:
    • date, existing date object
    • int or float, see datetime
    • str, following formats work:
      • YYYY-MM-DD
      • int or float, see datetime
  • time fields can be:
    • time, existing time object
    • str, following formats work:
      • HH:MM[:SS[.ffffff]][Z or [±]HH[:]MM]]]
  • timedelta fields can be:
    • timedelta, existing timedelta object
    • int or float, assumed as seconds
    • str, following formats work:
      • [-][DD ][HH:MM]SS[.ffffff]
      • [±]P[DD]DT[HH]H[MM]M[SS]S (ISO 8601 format for timedelta)

Boolean

Будет ошибка валидации, если значение не одно из;

  • True or False
  • 0 or 1
  • a str which when converted to lower case is one of ‘0’, ‘off’, ‘f’, ‘false’, ‘n’, ‘no’, ‘1’, ‘on’, ‘t’, ‘true’, ‘y’, ‘yes’
  • a bytes which is valid (per the previous rule) when decoded to str

Calable

Позволяет передавать функцию и указывать, какой выход в ней ожидается. Валидация не проверяте типы аргументов функции, только то, что этот объект вызываемый.

Type

Когда мы должны проверить, что объект является производным от tyoe, т.е. классом, не инстансом.

Literal Type

Появился в #python 3.8 typing.Literal (или typing_extensions.Literal для 3.8). позволяет определить только специфичные значения литералов. Позволяет проверять одно или больше специфичных значений без использования валидаторов.

fromtypingimportLiteralfrompydanticimportBaseModel,ValidationErrorclassPie(BaseModel):flavor:Literal[‘apple’,’pumpkin’]Pie(flavor=’apple’)Pie(flavor=’pumpkin’)try:Pie(flavor=’cherry’)exceptValidationErrorase:print(str(e))

Пример с Union

fromtypingimportOptional,UnionfromtypingimportLiteralfrompydanticimportBaseModelclassDessert(BaseModel):kind:strclassPie(Dessert):kind:Literal[‘pie’]flavor:Optional[str]classApplePie(Pie):flavor:Literal[‘apple’]classPumpkinPie(Pie):flavor:Literal[‘pumpkin’]classMeal(BaseModel):dessert:Union[ApplePie,PumpkinPie,Pie,Dessert]print(type(Meal(dessert={‘kind’:’pie’,’flavor’:’apple’}).dessert).__name__)#> ApplePie
print(type(Meal(dessert={‘kind’:’pie’,’flavor’:’pumpkin’}).dessert).__name__)#> PumpkinPie
print(type(Meal(dessert={‘kind’:’pie’}).dessert).__name__)#> Pie
print(type(Meal(dessert={‘kind’:’cake’}).dessert).__name__)#> Dessert

Анотированные типы

Пример с NamedTuple

fromtypingimportNamedTuplefrompydanticimportBaseModel,ValidationErrorclassPoint(NamedTuple):x:inty:intclassModel(BaseModel):p:Pointprint(Model(p=(‘1′,’2’)))#> p=Point(x=1, y=2)

pydantic types

  • FilePath
  • DirectoryPath
  • EmailStr
  • NameEmail
  • PyObject
  • Color html/css цвет вот в таком формате. Поддерживается несколько методов конвертации.
  • Json Пример
  • PaymentCardNumber Пример
  • AnyUrl описание про урлы
  • AnyHttpUrl
  • HttpUrl
  • PostgresDsn
  • RedisDsn
  • stricturl
  • UUID1 и 2, 3, 4, 5
  • SecretBytes и т.д. если нужно скрыт часть инфы из логирования
  • IPvAnyAddress и т.д.
  • NegativeFloat и т.д.
  • несколько методов, которые принимают методы, содержащие другие типы

Constrained types

Ограничение типов по форматам, диапазонам и т.д. через приставку con*

fromdecimalimportDecimalfrompydanticimport(BaseModel,NegativeFloat,NegativeInt,PositiveFloat,PositiveInt,NonNegativeFloat,NonNegativeInt,NonPositiveFloat,NonPositiveInt,conbytes,condecimal,confloat,conint,conlist,conset,constr,Field,)classModel(BaseModel):lower_bytes:conbytes(to_lower=True)short_bytes:conbytes(min_length=2,max_length=10)strip_bytes:conbytes(strip_whitespace=True)lower_str:constr(to_lower=True)short_str:constr(min_length=2,max_length=10)regex_str:constr(regex=r’^apple (pie|tart|sandwich)$’)strip_str:constr(strip_whitespace=True)big_int:conint(gt=1000,lt=1024)mod_int:conint(multiple_of=5)pos_int:PositiveIntneg_int:NegativeIntnon_neg_int:NonNegativeIntnon_pos_int:NonPositiveIntbig_float:confloat(gt=1000,lt=1024)unit_interval:confloat(ge=0,le=1)mod_float:confloat(multiple_of=0.5)pos_float:PositiveFloatneg_float:NegativeFloatnon_neg_float:NonNegativeFloatnon_pos_float:NonPositiveFloatshort_list:conlist(int,min_items=1,max_items=4)short_set:conset(int,min_items=1,max_items=4)decimal_positive:condecimal(gt=0)decimal_negative:condecimal(lt=0)decimal_max_digits_and_places:condecimal(max_digits=2,decimal_places=2)mod_decimal:condecimal(multiple_of=Decimal(‘0.25’))bigger_int:int=Field(…,gt=10000)

Описание всех аргументов (методов) смотри там же

Есть еще несколько специфичных случаев и можно задавать свои типы.

Config

classSprints(BaseModel):user:Usersprints:List[Sprint]classConfig:orm_mode=True

  • title заголовко json схемы
  • anystr_strip_whitespace и т.д.
  • validate_all
  • extra забыть, применить или игнорировать экстра атрибуты при инциализации
  • allow_mutation для неизменяемых типов
  • frozen открывает дорогу к хешированию инстансов модели
  • orm_mode использовать как модель для ORM
  • alias_generator
  • schema_extra
  • json_loads кастомные ф-ии для json
  • json_dumps
  • json_encoders

Сконфигурировать можно глобально, вот так:

classBaseModel(PydanticBaseModel):classConfig:arbitrary_types_allowed=True

Hi there

I’m the creator of the Conngenial App.
I mainly work with Python and Flutter.

Github Profile Details

LostInDarkMath

⚡ Github Stats

LostInDarkMath
LostInDarkMath

Github Streaks

LostInDarkMath

Github Contribution Graph

Ashish Kumar Activity Graph

Github Achievements

LostInDarkMath

Pydantic to the rescue

We might consider using a pydantic
model for the input validation.

Minimal start

We can start out with the simplest form of a pydantic model, with field types:

from pydantic import BaseModel

class InterpolationSetting(BaseModel):
interpolation_factor: int
interpolation_method: str
interpolate_on_integral: bool

Pydantic models are simply classes inheriting from the BaseModel class. We can create an instance of the new class as:

InterpolationSetting(
interpolation_factor=2,
interpolation_method=”linear”,
interpolate_on_integral=True
)

This automatically does two of the checks we had implemented:

  • interpolation_factor is an int
  • interpolate_on_integral is a bool

In the original script, the fields are in fact optional, i.e., it is possible to
provide no interpolation settings, in which case we do not do interpolation. We will
set the fields to optional later, and then implement the additional necessary checks.

We can verify the checks we have enforced now by supplying non-valid input:

from pydantic import ValidationError

try:
InterpolationSetting(
interpolation_factor=”text”,
interpolation_method=”linear”,
interpolate_on_integral=True,
)
except ValidationError as e:
print(e)

which outputs:

1 validation error for InterpolationSetting
interpolation_factor
value is not a valid integer (type=type_error.integer)

Pydantic raises a ValidationError when the validation of the model fails, stating
which field, i.e. attribute, raised the error and why. In this case
interpolation_factor raised a
type error because the value “text” is not a valid integer. The validation is
performed on instantiation of an InterpolationSetting object.

Validation of single fields and combinations of fields

Our original code also had some additional requirements:

  • interpolation_factor should be greater than or equal to two.
  • interpolation_method must be chosen from a set of valid methods.
  • We do not allow the combination of interpolate_on_integral=False
    and interpolation_method=”distribute”

The first restriction can be implemented using pydantic types. Pydantic provides many different types, we will use a constrained types this requirement, namely conint, a constrained integer type providing automatic restrictions such as lower limits.

The remaining two restrictions can be implemented as validators . We decorate our validation
functions with the validator decorator. The input argument to the validator decorator is the name of the attribute(s)
to perform the validation for.

All validators are run automatically when we instantiate
an object of the InterpolationSetting class, as for the type checking.

Our
validation functions are class methods, and the first argument is the class,
not an instance of the class. The second argument is the value to validate, and
can be named as we wish. We implement two validators, method_is_valid and valid_combination_of_method_and_on_integral:

from typing import Dict

from pydantic import BaseModel, conint, validator, root_validator

class InterpolationSetting(BaseModel):
interpolation_factor: conint(gt=1)
interpolation_method: str
interpolate_on_integral: bool

@validator(“interpolation_method”)
def method_is_valid(cls, method: str) -> str:
allowed_set = {“repeat”, “distribute”, “linear”, “cubic”, “akima”}
if method not in allowed_set:
raise ValueError(f”must be in {allowed_set}, got ‘{method}'”)
return method

@root_validator()
def valid_combination_of_method_and_on_integral(cls, values: Dict) -> Dict:
on_integral = values.get(“interpolate_on_integral”)
method = values.get(“interpolation_method”)
if on_integral is False and method == “distribute”:
raise ValueError(
f”Invalid combination of interpolation_method ”
f”{method} and interpolate_on_integral {on_integral}”
)
return values

There are a few things to note here:

  • Validators should return a validated value. The validators are run
    sequentially, and populate the fields of the data model if they are valid.
  • Validators should only raiseValueError, TypeError or AssertionError.
    Pydantic will catch these errors to populate the ValidationError and raise
    one exception regardless of the number of errors found in validation. You can read
    more about error handling
    in the docs.
  • When we validate a field against another, we can use the root_validator, which
    runs validation on entire model. Root validators are a little different: they have
    access to the values argument, which
    is a dictionary containing all fields that have already been validated. When the
    root validator runs, the interpolation_method may have failed to validate, in
    which case it will not be added to the values dictionary. Here, we handle that
    by using values.get(“interpolation_method”) which returns None if the key is
    not in values. The docs contain more information on root
    validators
    and
    field ordering,
    which is important to consider when we are using the values dictionary.

Again, we can verify by choosing input parameters to trigger the errors:

from pydantic import ValidationError

try:
InterpolationSetting(
interpolation_factor=1,
interpolation_method=”distribute”,
interpolate_on_integral=False,
)
except ValidationError as e:
print(e)

which outputs:

2 validation errors for InterpolationSetting
interpolation_factor
ensure this value is greater than 1 (type=value_error.number.not_gt; limit_value=1)
__root__
Invalid combination of interpolation_method distribute and interpolate_on_integral False (type=value_error)

As we see, pydantic raises a single ValidationError regardless of the number of ValueErrors raised in our model.

Implementing dynamic defaults

We also had some default values if certain parameters were not given:

  • If an interpolation_factor is given, set the default value linear
    for interpolation_method if none is given.
  • If an interpolation_factor is given, set the default value False
    for interpolate_on_integral if none is given.

In this case, we have dynamic defaults dependent on other fields.

This can also be achieved with root validators, by returning a conditional value.
As this means validating one field against another, we must take care to ensure
our code runs whether or not the two fields have passed validation and been added to
the values dictionary. We will now also use
Optional types, because we will handle the cases where not all values are provided. We add the new validators set_method_given_interpolation_factor and set_on_integral_given_interpolation_factor:

from typing import Dict, Optional

from pydantic import BaseModel, conint, validator, root_validator

class InterpolationSetting(BaseModel):
interpolation_factor: Optional[conint(gt=2)] interpolation_method: Optional[str] interpolate_on_integral: Optional[bool]

@validator(“interpolation_method”)
def method_is_valid(cls, method: Optional[str]) -> Optional[str]:
allowed_set = {“repeat”, “distribute”, “linear”, “cubic”, “akima”}
if method is not None and method not in allowed_set:
raise ValueError(f”must be in {allowed_set}, got ‘{method}'”)
return method

@root_validator()
def valid_combination_of_method_and_on_integral(cls, values: Dict) -> Dict:
on_integral = values.get(“interpolate_on_integral”)
method = values.get(“interpolation_method”)
if on_integral is False and method == “distribute”:
raise ValueError(
f”Invalid combination of interpolation_method ”
f”{method} and interpolate_on_integral {on_integral}”
)
return values

@root_validator()
def set_method_given_interpolation_factor(cls, values: Dict) -> Dict:
factor = values.get(“interpolation_factor”)
method = values.get(“interpolation_method”)
if method is None and factor is not None:
values[“interpolation_method”] = “linear”
return values

@root_validator()
def set_on_integral_given_interpolation_factor(cls, values: Dict) -> Dict:
on_integral = values.get(“interpolate_on_integral”)
factor = values.get(“interpolation_factor”)
if on_integral is None and factor is not None:
values[“interpolate_on_integral”] = False
return values

We can verify that the default values are set only when interpolation_factor
is provided, running InterpolationSetting(interpolation_factor=3) returns:

InterpolationSetting(interpolation_factor=3, interpolation_method=’linear’, interpolate_on_integral=None)

whereas supplying no input parameters, InterpolationSetting(), returns a data model with all parameters set to None:

InterpolationSetting(interpolation_factor=None, interpolation_method=None, interpolate_on_integral=None)

Note: If we have static defaults, we can simply set them for the fields:

class InterpolationSetting(BaseModel):
interpolation_factor: Optional[int] = 42

Final safeguard against typos

Finally, we had one more check in out previous script: That no unknown keys were provided. If we provide unknown keys to our data model now, nothing really happens, for example InterpolationSetting(hello=”world”) outputs:

InterpolationSetting(interpolation_factor=None, interpolation_method=None, interpolate_on_integral=None)

Often, an unknown field name is the result of a typo
in the toml file. Therefore we want to raise an error to alert the user.
We do this using a the model config, controlling the behaviour of the model. The extra attribute of the config determines what we do with extra fields. The default is ignore, which we can see in the example above, where the field is ignored, and not added to the model, as the option allow does. We can use the forbid option to raise an exception when extra fields are supplied.

from typing import Dict, Optional

from pydantic import BaseModel, conint, validator, root_validator

class InterpolationSetting(BaseModel):
interpolation_factor: Optional[conint(gt=2)] interpolation_method: Optional[str] interpolate_on_integral: Optional[bool]

class Config:
extra = “forbid”

@validator(“interpolation_method”)
def method_is_valid(cls, method: Optional[str]) -> Optional[str]:
allowed_set = {“repeat”, “distribute”, “linear”, “cubic”, “akima”}
if method is not None and method not in allowed_set:
raise ValueError(f”must be in {allowed_set}, got ‘{method}'”)
return method

@root_validator()
def valid_combination_of_method_and_on_integral(cls, values: Dict) -> Dict:
on_integral = values.get(“interpolate_on_integral”)
method = values.get(“interpolation_method”)
if on_integral is False and method == “distribute”:
raise ValueError(
f”Invalid combination of interpolation_method ”
f”{method} and interpolate_on_integral {on_integral}”
)
return values

@root_validator()
def set_method_given_interpolation_factor(cls, values: Dict) -> Dict:
factor = values.get(“interpolation_factor”)
method = values.get(“interpolation_method”)
if method is None and factor is not None:
values[“interpolation_method”] = “linear”
return values

@root_validator()
def set_on_integral_given_interpolation_factor(cls, values: Dict) -> Dict:
on_integral = values.get(“interpolate_on_integral”)
factor = values.get(“interpolation_factor”)
if on_integral is None and factor is not None:
values[“interpolation_factor”] = False
return values

If we try again with an unknown key, we now get a ValidationError:

from pydantic import ValidationError

try:
InterpolationSetting(hello=True)
except ValidationError as e:
print(e)

This raises a validation error for the unknown field:

1 validation error for InterpolationSetting
hello
extra fields not permitted (type=value_error.extra)

Adapting our existing code is easy

Now we have implemented all our checks, and can go on to adapt our existing code to use the new data model. In our original implementation, we would do something like

params_in = toml.load(path_to_settings_file)
params_validated = validate_input_settings(params_in)
interpolate_result(params_validated)

We can replace the call to validate_input_settings with instantiation of the pydantic model: params_validated = InterpolationSetting(params_in). Each pydantic data model has a .dict() method that returns the parameters as a dictionary, so we can use it in the input argument to interpolate_result directly: interpolate_result(params_validated.dict()). Another option is to refactor interpolate_result to use the attributes of the InterpolationSetting objects, such as params_validated.interpolation_method instead of the values of a dictionary.

BaseModel

Чтобы получить доступ к дополнительным возможностям таким как сериализация (Serialization)
и поддержка JSON воспользуемся классом BaseModel

Просто напомню, что сперва у нас было

from dataclasses import dataclass

Затем

frompydantic.dataclasses import dataclass

А сейчас нужно сделать

from pydantic import BaseModel

И убрать декоратор @dataclass перед class IceCreamMix:

class IceCreamMix: нужно заменить на class IceCreamMix(BaseModel):

а также добавить имена аттрибутов код, создающий объект класса IceCreamMix
то есть name = “PB&J” flavor = Flavor.peanut_butter и так далее

strawberries = ‘strawberries’classIceCreamMix(BaseModel): name: str flavor: Flavor toppings: Tuple[Topping, …] scoops: intdefmain(): ice_cream_mix = IceCreamMix(name = “PB&J”,flavor = Flavor.peanut_butter,toppings = (Topping.strawberries, Topping.sprinkles),scoops = 2 )

python PydanticDemo.py

IceCreamMix(name=’PB&J’, flavor=, toppings=(, ), scoops=2)

Всё работает так же, как и до изменений.

Теперь можно вывести результат в виде JSON

print(ice_cream_mix.json())

python PydanticDemo.py

{“name”: “PB&J”, “flavor”: “peanut butter”, “toppings”: [“strawberries”, “brownie”], “scoops”: 2}

Обратите внимание на JSON который вы получили выше.

Его можно скопировать, затем если нужно изменить и создать ещё один объект
прямо из JSON с помощью метода parse_raw()
Например:

another_mix = IceCreamMix.parse_raw(‘{“name”: “New mix”, “flavor”: “mint”, “toppings”: [“cookies”, “hot fudge”], “scoops”: 2}’)print(another_mix.json())

{“name”: “New mix”, “flavor”: “mint”, “toppings”: [“cookies”, “hot fudge”], “scoops”: 2}

Если случайно ошибиться со значением аттрибута – pydantic не даст соврать

another_mix = IceCreamMix.parse_raw(‘{“name”: “New mix”, “flavor”: “novichoke”, “toppings”: [“cookies”, “hot fudge”], “scoops”: 2}’)print(another_mix.json())

python PydanticDemo.py

Traceback (most recent call last):
File “/home/avorotyn/python/pydantic/PydanticDemo.py”, line 45, in
main()
File “/home/avorotyn/python/pydantic/PydanticDemo.py”, line 40, in main
another_mix = IceCreamMix.parse_raw(‘{“name”: “New mix”, “flavor”: “novichoke”, “toppings”: [“cookies”, “hot fudge”], “scoops”: 2}’)
File “pydantic/main.py”, line 543, in pydantic.main.BaseModel.parse_raw
File “pydantic/main.py”, line 520, in pydantic.main.BaseModel.parse_obj
File “pydantic/main.py”, line 362, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for IceCreamMix
flavor
value is not a valid enumeration member; permitted: ‘chocolate’, ‘vanilla’, ‘strawberry’, ‘mint’, ‘coffee’, ‘peanut butter’ (type=type_error.enum; enum_values=[, , , , , ])

Пример использования

Рассмотрим скрипт

PydanticDemo.py

from dataclasses import dataclassfrom typing import Tuplefrom enum import Enum@dataclassclassIceCreamMix: name: str flavor: str toppings: Tuple[str, …] scoops: intdefmain(): ice_cream_mix = IceCreamMix( “PB&J”, “peanut butter”, (“strawberries”, “sprinkles”), 2 )print(ice_cream_mix)if__name__ == ‘__main__’: main()

python PydanticDemo.py

IceCreamMix(name=’PB&J’, flavor=’peanut butter’, toppings=(‘strawberries’, ‘sprinkles’), scoops=2)

Этот скрипт успешно демонстрирует тип мороженого

Добавим ещё немного ООП

from dataclasses import dataclassfrom typing import Tuplefrom enum import EnumclassFlavor(str, Enum): chocolate = ‘chocolate’ vanilla = ‘vanilla’ strawberry = ‘strawberry’ mint = ‘mint’ coffeee = ‘coffee’ peanut_butter = ‘peanut butter’classTopping(str, Enum): sprinkles = ‘sprinkles’ hot_fudge = ‘hot fudge’ cookies = ‘cookies’ brownie = ‘brownie’ whipped_cream = ‘whipped cream’ strawberries = ‘strawberries’@dataclassclassIceCreamMix: name: str flavor: Flavor toppings: Tuple[Topping, …] scoops: intdefmain(): ice_cream_mix = IceCreamMix( “PB&J”, Flavor.peanut_butter, (Topping.strawberries, Topping.sprinkles), 2 )print(ice_cream_mix)if__name__ == ‘__main__’: main()

$ python PydanticDemo.py

IceCreamMix(name=’PB&J’, flavor=, toppings=(, ), scoops=2)

Скрипт по-прежнему работает

Что если мы по ошибке выберем несуществующий запах или топпинг

defmain(): ice_cream_mix = IceCreamMix( “PB&J”,”smells like shit”, (Topping.strawberries, 111), 2 )

python PydanticDemo.py

IceCreamMix(name=’PB&J’, flavor=’smells like shit’, toppings=(, 111), scoops=2)

Скрипт не замечает подвоха.

Чтобы проверять данные автоматически установите pydantic и внесите всего одно изменение
в первую строку

frompydantic.dataclasses import dataclass

python PydanticDemo.py

Traceback (most recent call last):
File “PydanticDemo.py”, line 41, in
main()
File “PydanticDemo.py”, line 31, in main
ice_cream_mix = IceCreamMix(
File ““, line 7, in __init__
File “C:UsersAndreipythonpydanticvenvlibsite-packagespydanticdataclasses.py”, line 99, in _pydantic_post_init
raise validation_error
pydantic.error_wrappers.ValidationError: 2 validation errors for IceCreamMix
flavor
value is not a valid enumeration member; permitted: ‘chocolate’, ‘vanilla’, ‘strawberry’, ‘mint’, ‘coffee’, ‘peanut butter’ (type=type_error.enum; enum_values=[, , , , , ])
toppings -> 1
value is not a valid enumeration member; permitted: ‘sprinkles’, ‘hot fudge’, ‘cookies’, ‘brownie’, ‘whipped cream’, ‘strawberries’ (type=type_error.enum; enum_values=[, , , , , ])

pydantic не пропустил наш код. Разберём выдачу подробнее

pydantic.error_wrappers.ValidationError: 2 validation errors for IceCreamMix

Указано количество ошибок и класс. Это помогло бы с дебагом, если бы мы не знали заранее где ошибки
и сколько их

flavor
value is not a valid enumeration member; permitted:

‘chocolate’,
‘vanilla’,
‘strawberry’,
‘mint’,
‘coffee’,
‘peanut butter’

(type=type_error.enum; enum_values=[, , , , , ])

Pydantic подсказывает допустимые значения.

Тоже самое и с топпингами, где вместо допустимого значения стоит 111

toppings -> 1
value is not a valid enumeration member; permitted: ‘sprinkles’, ‘hot fudge’, ‘cookies’, ‘brownie’, ‘whipped cream’, ‘strawberries’ (type=type_error.enum; enum_values=[, , , , , ])

Верните корректные значения для Flavor и Topping но замените scoops с 2 на ‘2’

defmain(): ice_cream_mix = IceCreamMix( “PB&J”, Flavor.peanut_butter, (Topping.strawberries, Topping.sprinkles),’2′ )

python PydanticDemo.py

IceCreamMix(name=’PB&J’, flavor=, toppings=(, ), scoops=2)

scoops по-прежнему равно двум

Pydantic поддерживает приведение типа (type coercion)

Экспорт моделей в другие форматы данных

pedantic-python-decorators’s People

Similar Articles

Nested Scroll Music Player App in Jetpack Compose

Languages, API

Creating a Nested Scroll Music Player App in Jetpack Compose

Read MoreRazorpay React-Django Application example Image

API

Integrating Razorpay in a React-Django Application

Read MoreImplementing a Spring Query Language

API

Implementing a Spring Query Language Spring Boot Search

Read More

Company

  • About
  • Careers
  • Legals

Resources

  • Blog
  • Content Library
  • Engineering Education

Conversion, data validation and schemas

Converting raw data to Python objects actually involves two steps, as the following figure illustrates:

Data serialization flowContributors

dependabot[bot] avatar
lostindarkmath avatar

Stargazers

 avatar
 avatar
 avatar
 avatar
 avatar
 avatar
 avatar
 avatar
 avatar
 avatar
 avatar
 avatar
 avatar
 avatar
 avatar
 avatar
 avatar
 avatar

Watchers

 avatar
 avatar

pedantic-python-decorators’s Issues

coin flip TypeHinting-Errors at Generic Types

Hey there,
I upgradet to Pedantic 1.20 but came across a severe side effect error in pedantic.
All of my ~130 tests were successful in version<1.20 but now in 1.20 there are 4 failing tests.
All of them refer to this function of my stack-class:

deftop(self) ->Optional[T]:
iflen(self.items) >0:
returnself.items[len(self.items)-1] else:
returnNone

When the stack is empty, I want to return None, which I typehint with Optional.
Sometimes pedantic accepts this function and sometimes not! It depends on the order of my unit tests… Look at the two pictures:
pedantic_test1

If I put the failed test at first, other tests fail:
pedantic_test2

If I execute my test solely, everything is fine.
pedantic_test3

All of my tests are independent from each other. If I follow the debugger, sometimes pedantic throws an error and sometimes not – with everything else being equal.

Here is the full implementation of my stack class:

fromtypingimportList, Optional, UnionfromtypingimportTypeVar, Genericfrompedanticimportpedantic_classT=TypeVar(‘T’)

@pedantic_classclassStack(Generic[T]):
def__init__(self) ->None:
self.items: List[T] = []

defpush(self, item: T) ->None:
self.items.append(item)

defpop(self) ->T:
returnself.items.pop()

defempty(self) ->bool:
returnnotself.itemsdeftop(self) ->Optional[T]:
iflen(self.items) >0:
returnself.items[len(self.items)-1] else:
returnNone

So far, in all of every tests I instantiate a new stack object with an empty list of items. And I only put igraph.Vertex objects in it.

……
I spend some time in the debugger understanding this behaviour:
The first time I call stack.top() and let it return a None, everything is fine.
Second, third and n-th time returning a None will be fine.
But then the first time calling stack.top() when expecting to return a igraph.Vertex(), pedantic screams at me:

AssertionError: InfunctionStack.top:
EForTypeVar~Texistsatypeconflict: valueNonehastypeandvalueigraph.Vertex(, 0, {‘name’: ‘open’}) hastype

Pedantic expects a None.
So it looks like, your inner pedantic workings bind ‘T’ only to a None and not to None and the runtime-T-type.
Hope you can fix it.

Pydantic and JSON features

We can convert the Pydantic model to a JSON string using the json() function:

print(blog.json())

# {“title”: “Hello World”, “author”: “Saransh Kataria”, “categories”: [“Wisdom”, “Geek”]}

And we can parse a JSON to a Pydantic model using the parse_raw function:

blog = Blog.parse_raw(‘{“title”: “Hello World”, “author”: “Saransh Kataria”, “categories”: [“Wisdom”, “Geek”]}’)
print(blog.title)

# Hello World

And all of the validations will be performed while doing the JSON parsing. And if there are any errors during parsing, ValidationError with friendly messages will be thrown for those.

Источники

  • https://www.augmentedmind.de/2020/10/25/marshmallow-vs-pydantic-python/
  • https://www.wisdomgeek.com/development/web-development/python/parsing-and-validating-data-in-python-using-pydantic/
  • https://konstantinklepikov.github.io/myknowlegebase/notes/pydantic.html
  • https://coder.social/LostInDarkMath/pedantic-python-decorators
  • https://datascience.statnett.no/2020/05/11/how-we-validate-data-using-pydantic/
  • https://www.AndreyOlegovich.ru/code/python/pydantic/
  • https://www.section.io/engineering-education/strict-type-validation-with-pydantic/
[свернуть]
Решите Вашу проблему!


×
Adblock
detector