-
Notifications
You must be signed in to change notification settings - Fork 332
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Python: Add 'factory' support #1336
Python: Add 'factory' support #1336
Conversation
There are the config and module which I used to test {
"applications": {
"python-app": {
"type": "python",
"processes": 2,
"path": "/www/",
"targets": {
"front": {
"module": "app",
"callable": "app_factory",
"factory": true
},
"back": {
"module": "app",
"callable": "app"
},
"back_door": {
"module": "app",
"callable": "app",
"factory": false
}
}
}
},
"listeners": {
"127.0.0.1:8100": {
"pass": "applications/python-app/front"
},
"127.0.0.1:8200": {
"pass": "applications/python-app/back"
},
"127.0.0.1:8300": {
"pass": "applications/python-app/back_door"
}
}
} Python module: def app(environ, start_response):
start_response('200 OK', [('Content-Type', 'text/html')])
return [b'Hell WSGI', b'Hello world']
def application_f(environ, start_response):
start_response('200 OK', [('Content-Type', 'text/html')])
return [b'from app created from factory']
def app_factory():
return application_f |
OK, I see this is now taking a completely different tact... |
sure will take care of this. actually, the changes were very much different from each other so thought of creating new pr itself |
@ac000 changes made as suggested |
@gourav-kandoria Thanks! Let me squash those commits down and see what we end up with.. I'll push up the result for you to check... |
Hmm, looking at the static nxt_conf_vldt_object_t nxt_conf_vldt_python_members[] = {
{
.name = nxt_string("module"),
.type = NXT_CONF_VLDT_STRING,
.validator = nxt_conf_vldt_targets_exclusive,
.u.string = "module",
}, {
.name = nxt_string("callable"),
.type = NXT_CONF_VLDT_STRING,
.validator = nxt_conf_vldt_targets_exclusive,
.u.string = "callable",
}, {
.name = nxt_string("factory"),
.type = NXT_CONF_VLDT_BOOLEAN
}, {
.name = nxt_string("prefix"),
.type = NXT_CONF_VLDT_STRING,
.validator = nxt_conf_vldt_targets_exclusive,
.u.string = "prefix",
}, {
.name = nxt_string("targets"),
.type = NXT_CONF_VLDT_OBJECT,
.validator = nxt_conf_vldt_targets,
.u.members = nxt_conf_vldt_python_target_members
},
NXT_CONF_VLDT_NEXT(nxt_conf_vldt_python_common_members)
}; It seems none of these options can be used when "targets" is specified, so I will go ahead and add the I also need to check what the |
OK, so I'm pretty sure the |
1144039
to
ca04789
Compare
@gourav-kandoria Please check over the updated patch. Please also check over my attempt at explaining this in the commit message. I think I'm still missing a concrete reason why this is useful. Commit message and diff of my changes
diff --git ./src/nxt_conf_validation.c ./src/nxt_conf_validation.c
index ed3aa735..96592652 100644
--- ./src/nxt_conf_validation.c
+++ ./src/nxt_conf_validation.c
@@ -840,7 +840,9 @@ static nxt_conf_vldt_object_t nxt_conf_vldt_python_members[] = {
.u.string = "callable",
}, {
.name = nxt_string("factory"),
- .type = NXT_CONF_VLDT_BOOLEAN
+ .type = NXT_CONF_VLDT_BOOLEAN,
+ .validator = nxt_conf_vldt_targets_exclusive,
+ .u.string = "factory",
}, {
.name = nxt_string("prefix"),
.type = NXT_CONF_VLDT_STRING,
diff --git ./src/python/nxt_python.c ./src/python/nxt_python.c
index 9cb62878..4a08cc64 100644
--- ./src/python/nxt_python.c
+++ ./src/python/nxt_python.c
@@ -403,6 +403,7 @@ nxt_python_set_target(nxt_task_t *task, nxt_python_target_t *target,
char *callable, *module_name;
PyObject *module, *obj;
nxt_str_t str;
+ nxt_bool_t is_factory = 0;
nxt_conf_value_t *value;
static nxt_str_t module_str = nxt_string("module");
@@ -451,25 +452,27 @@ nxt_python_set_target(nxt_task_t *task, nxt_python_target_t *target,
}
value = nxt_conf_get_object_member(conf, &factory_flag_str, NULL);
- nxt_bool_t is_factory = 0;
if (value != NULL) {
is_factory = nxt_conf_get_boolean(value);
}
+
if (is_factory) {
if (nxt_slow_path(PyCallable_Check(obj) == 0)) {
- nxt_alert(task, "factory \"%s\" in module \"%s\" can ",
- "not be called to fetch callable",
- callable, module_name);
+ nxt_alert(task,
+ "factory \"%s\" in module \"%s\" ",
+ "can not be called to fetch callable",
+ callable, module_name);
goto fail;
}
obj = PyObject_CallObject(obj, NULL);
if (nxt_slow_path(PyCallable_Check(obj) == 0)) {
- nxt_alert(task, "factory \"%s\" in module \"%s\" ",
- "did not return callable object",
- callable, module_name);
+ nxt_alert(task,
+ "factory \"%s\" in module \"%s\" ",
+ "did not return callable object", callable, module_name);
goto fail;
}
+
} else if (nxt_slow_path(PyCallable_Check(obj) == 0)) {
nxt_alert(task, "\"%s\" in module \"%s\" is not a callable object",
callable, module_name); |
@ac000 I'm happy to take a crack at explaining it this afternoon. The factory stuff will make sense to Pythonistas. |
The Flask docs call out two main use cases:
So if you've structured your application in that way, for those reasons, it'd be really handy to be able to run it in Unit as well :) |
@ac000 here's my crack at a commit message:
|
ca04789
to
cf8620e
Compare
|
Does this warrant adding a test case for the new Python language module "factory" boolean setting, it defaults to We'll also need to update the docs... |
Adds support for the app factory pattern to the Python language module. A factory is a callable that returns a WSGI or ASGI application object. Unit does not support passing arguments to factories. Setting the `factory` option to `true` instructs Unit to treat the configured `callable` as a factory. For example: "my-app": { "type": "python", "path": "/srv/www/", "module": "hello", "callable": "create_app", "factory": true } This is similar to other WSGI / ASGI servers. E.g., $ uvicorn --factory hello:create_app $ gunicorn 'hello:create_app()' The factory setting defaults to false. Closes: nginx#1106 Link: <nginx#1336 (comment)> [ Commit message - Dan / Minor code tweaks - Andrew ] Signed-off-by: Andrew Clayton <[email protected]>
cf8620e
to
fcf590c
Compare
|
Definitely needs docs, and I'd strongly prefer at least a simple WSGI test to be included in this patch before we merge. |
@callahad, I guess, It should be better, if we are able to give support of arguments to pass to factory, Because, if someone is using factory, they might be using it for the purpose of being able to get any specific callable from the factory. Otherwise, factory without arguments is functionally equivalent to just callable option. |
I was wondering why you couldn't just have your callable call something else... |
well, the information regarding what to call has to come from somewhere. so factory will return the callable with this info preconfigured. So, I think that's a choice, one may prefer one over the other. |
Gunicorn is the only server that supports passing arguments to factories, and even its docs suggest using environment variables instead of args. I think there's value in merging this as-is, without argument support. That gives us parity with uvicorn, uWSGI, and gunicorn's preferred approach to factories. |
No worries, you can leave it as is (in |
Looks like there is some error in the new test somewhere, care to take a look? |
Adds support for the app factory pattern to the Python language module. A factory is a callable that returns a WSGI or ASGI application object. Unit does not support passing arguments to factories. Setting the `factory` option to `true` instructs Unit to treat the configured `callable` as a factory. For example: "my-app": { "type": "python", "path": "/srv/www/", "module": "hello", "callable": "create_app", "factory": true } This is similar to other WSGI / ASGI servers. E.g., $ uvicorn --factory hello:create_app $ gunicorn 'hello:create_app()' The factory setting defaults to false. Closes: nginx#1106 Link: <nginx#1336 (comment)> [ Commit message - Dan / Minor code tweaks - Andrew ] Signed-off-by: Andrew Clayton <[email protected]>
…fig" Add the following tests cases: 1. When "factory" key is used inside the "targets" option. 2. When "factory" key is used at the root level of python application config. 3. When factory returns invalid callable or When factory is invalid callable Closes: nginx#1106 Link: <nginx#1336> Signed-off-by: Andrew Clayton <[email protected]>
8185fe9
to
72e6d16
Compare
…fig" Add the following tests cases: 1. When "factory" key is used inside the "targets" option. 2. When "factory" key is used at the root level of python application config. 3. When factory returns invalid callable or When factory is invalid callable Closes: nginx#1106 Link: <nginx#1336> Signed-off-by: Andrew Clayton <[email protected]>
72e6d16
to
be3d582
Compare
Adds support for the app factory pattern to the Python language module. A factory is a callable that returns a WSGI or ASGI application object. Unit does not support passing arguments to factories. Setting the `factory` option to `true` instructs Unit to treat the configured `callable` as a factory. For example: "my-app": { "type": "python", "path": "/srv/www/", "module": "hello", "callable": "create_app", "factory": true } This is similar to other WSGI / ASGI servers. E.g., $ uvicorn --factory hello:create_app $ gunicorn 'hello:create_app()' The factory setting defaults to false. Closes: nginx#1106 Link: <nginx#1336 (comment)> [ Commit message - Dan / Minor code tweaks - Andrew ] Signed-off-by: Andrew Clayton <[email protected]>
Add the following tests cases: 1. When "factory" key is used inside the "targets" option. 2. When "factory" key is used at the root level of python application config. 3. When factory returns invalid callable or When factory is invalid callable Link: <nginx#1336> [ Commit subject & message formatting tweaks - Andrew ] Signed-off-by: Andrew Clayton <[email protected]>
be3d582
to
5b06b11
Compare
That is what I'm planning to merge. This doesn't do if (nxt_fast_path(value == NULL)) { or if (nxt_slow_path(is_factory)) { As I simply don't see the point here. That is also passing judgement on peoples configuration and will "penalise" (but not really) people who do enable this feature... |
@ac000 I guess task seems to be completed now. Are there any steps pending, before merge can be done. |
test/python/factory/wsgi.py
Outdated
wsgi_invalid_callable = None | ||
|
||
def wsgi_factory_returning_invalid_callable(): | ||
return wsgi_invalid_callable |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
EOF please.
test/test_python_factory.py
Outdated
assert resp['body'] == '2' | ||
|
||
def test_python_factory_invalid_callable_value(skip_alert): | ||
skip_alert(r"[\s\S]*") # This regex will match to any string |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This will skip all alerts, including potentially dangerous ones (like segfaults), so if some alerts are expected, they should be listed explicitly:
- skip_alert(r"[\s\S]*") # This regex will match to any string
+ skip_alert(
+ r"failed to create initial state",
+ r"failed to apply new conf",
+ r"did not return callable object",
+ r"can not be called to fetch callable",
+ )
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @andrey-zelenkov just a question. on my local testing found only these alerts only
r"failed to apply new conf",
r"did not return callable object",
r"can not be called to fetch callable"
so will add skip_alert for them
Did you see it anywhere ("failed to create initial state",)
while running tests of this file, or you have added it only as an example here.
…fig" Add the following tests cases: 1. When "factory" key is used inside the "targets" option. 2. When "factory" key is used at the root level of python application config. 3. When factory returns invalid callable or When factory is invalid callable Closes: nginx#1106 Link: <nginx#1336> Signed-off-by: Andrew Clayton <[email protected]>
5b06b11
to
cd063f7
Compare
Adds support for the app factory pattern to the Python language module. A factory is a callable that returns a WSGI or ASGI application object. Unit does not support passing arguments to factories. Setting the `factory` option to `true` instructs Unit to treat the configured `callable` as a factory. For example: "my-app": { "type": "python", "path": "/srv/www/", "module": "hello", "callable": "create_app", "factory": true } This is similar to other WSGI / ASGI servers. E.g., $ uvicorn --factory hello:create_app $ gunicorn 'hello:create_app()' The factory setting defaults to false. Closes: nginx#1106 Link: <nginx#1336 (comment)> [ Commit message - Dan / Minor code tweaks - Andrew ] Signed-off-by: Andrew Clayton <[email protected]>
…fig" Add the following tests cases: 1. When "factory" key is used inside the "targets" option. 2. When "factory" key is used at the root level of python application config. 3. When factory returns invalid callable or When factory is invalid callable Closes: nginx#1106 Link: <nginx#1336> Signed-off-by: Andrew Clayton <[email protected]>
cd063f7
to
688f1c7
Compare
LGTM. |
Adds support for the app factory pattern to the Python language module. A factory is a callable that returns a WSGI or ASGI application object. Unit does not support passing arguments to factories. Setting the `factory` option to `true` instructs Unit to treat the configured `callable` as a factory. For example: "my-app": { "type": "python", "path": "/srv/www/", "module": "hello", "callable": "create_app", "factory": true } This is similar to other WSGI / ASGI servers. E.g., $ uvicorn --factory hello:create_app $ gunicorn 'hello:create_app()' The factory setting defaults to false. Closes: nginx#1106 Link: <nginx#1336 (comment)> [ Commit message - Dan / Minor code tweaks - Andrew ] Signed-off-by: Andrew Clayton <[email protected]>
Add the following tests cases: 1. When "factory" key is used inside the "targets" option. 2. When "factory" key is used at the root level of python application config. 3. When factory returns invalid callable or When factory is invalid callable Link: <nginx#1336> [ Commit subject & message formatting tweaks - Andrew ] Signed-off-by: Andrew Clayton <[email protected]>
688f1c7
to
d67d350
Compare
|
@gourav-kandoria Thanks! and thanks for your patience working through the review process... |
I'll second that; a sincere thank you for your work and engagement, @gourav-kandoria! |
This pr adds the factory support in python languange. By introducing new option as "factory" which can be set as true or false, if true, the callable will be interpreted as factory otherwise not