Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨ Migrate from Pydantic V1 to V2 [WIP] #76

Merged
merged 27 commits into from
Jun 9, 2024
Merged

✨ Migrate from Pydantic V1 to V2 [WIP] #76

merged 27 commits into from
Jun 9, 2024

Conversation

DavidRomanovizc
Copy link

@DavidRomanovizc DavidRomanovizc commented Jan 31, 2024

What was wrong?

Need to migrate project from Pydantic V1 to V2
Related to Issue: #51

Todo:

  • Update documentation

@DavidRomanovizc DavidRomanovizc changed the title ✨ Migrate from Pydantic V1 to V2. ✨ Migrate from Pydantic V1 to V2 [WIP] Feb 11, 2024
@mahenzon
Copy link
Member

Hey there! Please run pre-commit locally. Thank you

@DavidRomanovizc
Copy link
Author

@mahenzon could you review my PR, please? If you're not busy with something else, I'd appreciate your feedback

examples/api_minimal.py Outdated Show resolved Hide resolved
@@ -33,7 +32,7 @@
RELATIONSHIP_SPLITTER = "."

# The mapping with validators using by to cast raw value to instance of target type
REGISTERED_PYDANTIC_TYPES: Dict[Type, List[Callable]] = dict(_VALIDATORS)
REGISTERED_PYDANTIC_TYPES: Dict[Type, List[Callable]] = {}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we get rid of _VALIDATORS? and would it work?

else:
return result_value, errors
try:
# Создаем экземпляр схемы для валидации значения
Copy link
Member

@mahenzon mahenzon Mar 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

я пытался пойти таким путём в v1, там не проканало со сложными типами с доп валидаторами, например список из строк, а в валидаторе мы его создаём, если не передано, и что-то подобное. тут надо будет в тестах зафиксировать, что всё ок

@@ -318,7 +312,7 @@ def get_operator(model_column: InstrumentedAttribute, operator_name: str) -> str


def get_custom_filter_expression_callable(schema_field, operator: str) -> Callable:
return schema_field.field_info.extra.get(
return schema_field.default.extra.get(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

разве есть extra на default?

@mahenzon
Copy link
Member

Looks great. Please rebase + fix tests + resolve comments

@mbp101
Copy link

mbp101 commented Apr 23, 2024

@DavidRomanovizc, thanks for your work on this. When do you think you might be able to review/incorporate @mahenzon's feedback?

@mahenzon
Copy link
Member

mahenzon commented May 1, 2024

@DavidRomanovizc thank you for all your hard work! Please resolve conflicts (rebase on the target branch dev-3.x) 🙏

tests/models.py Show resolved Hide resolved
validated = AtomicOperationRequest.parse_obj(operation_request)
assert validated.dict(exclude_unset=True, by_alias=True) == operation_request
validated = AtomicOperationRequest.model_validate(operation_request)
assert validated.model_dump(exclude_unset=True, by_alias=True) == operation_request

def test_not_supported_operation(
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ValidationError now returns a new error message but I'm uncertain if we should modify the test itself to accommodate this change. I couldn't find where pydantic sets this new error message, which makes me hesitant to update the test without understanding the reason behind the change

@mahenzon
Copy link
Member

mahenzon commented May 6, 2024

Hey there! Thank you again.
Now tests are failing because of broken pydantic schemas, please take a look:

current PR branch:
Screenshot 2024-05-06 at 11 03 42
We can see, that schema in the signature has relationships fields (but it should not)

here's the same test in the main branch:
Screenshot 2024-05-06 at 11 08 32
no relationships schemas are present.

this is the reason why tests are failing with "Param: posts can only be a request body, using Body()". This param should not be even present.

Please check schemas generation 🙏

@@ -50,26 +48,21 @@ def create_additional_query_params(schema: Optional[Type[BaseModel]]) -> tuple[l
available_includes_names = []

# TODO! ?
schema.update_forward_refs(**registry.schemas)
for name, field in (schema.__fields__ or {}).items():
schema.model_rebuild(_types_namespace=registry.schemas)
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'am hesitant about this change, because now we get the pydantic.errors.PydanticUndefinedAnnotation: name 'PostSchema' is not defined which most likely means that model_rebuild works not as it should be or declared in the wrong place

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any other way to update string type annotations to real objects? The only reason to do it is the need to work with real pydantic schemas when building APIs

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a function that allows you to update type annotations to real obj. So far, it seems to be handling the task well, and I've placed it in schema_builder.py::string_to_schema

@@ -488,7 +490,7 @@ def create_jsonapi_object_schemas(
if use_schema_cache and schema in self.object_schemas_cache and includes is not_passed:
return self.object_schemas_cache[schema]

schema.update_forward_refs(**registry.schemas)
schema.model_rebuild(_types_namespace=registry.schemas)
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And here

@DavidRomanovizc
Copy link
Author

Also I commented out the keys_to_ids_list field for now, as it's causing an error again: Param: ... can only be a request body, using Body(). I checked the main branch and found that the field is in the schema's signature, but the type annotations are slightly different and I don't have any idea what the issue is yet.

@mahenzon
Copy link
Member

mahenzon commented Jun 9, 2024

Thank you a lot for your work! I'll take it from here

@mahenzon mahenzon merged commit f9295e7 into mts-ai:dev-3.x Jun 9, 2024
4 of 10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants