Flexibility in scheduling the job

[agllno]

• Any way to schedule one time job to run
• Is it possible to start the job immediately after scheduling, rather running it as delayed job?

If these options are already available , please update the documentation with relevant examples

[zengyan-amazon]

Hi agllno,

Currently we don't support one time job out of the box. You can consider implementing your job runner to update your job config to disable the job, or delete the job from your job index. Hope this helps.

[JackRyanson]

@zengyan-amazon we're also quite in need for the one time job scheduling, would be great to have these. Thanks

[agllno]

@zengyan-amazon can you please clarify on second point as well

[zengyan-amazon]

@agllno ,

sorry for missing the second point. the job scheduler doesn't support executing job immediately after scheduling. You can create a feature request for it, and also you are welcome to contribute to this feature.

Meanwhile, if you want to trigger something immediately, maybe you can consider implementing it as an API.

[dbbaughe]

Hi @agllno and @JackRyanson,

Curious what the use case is for one time job scheduling that you have.

Thanks,
Drew

[JackRyanson]

@dbbaughe one time data enrichment triggered on the user side. E.g. NLP this document set, or lenghty computations e.g. an advanced algorithm that produces results in an index.

[dbbaughe]

@JackRyanson,

Ah so a manually started job from user side?
Is your use case something like:
[User does something] -> [Creates job] -> [Triggers once and deletes itself]
Or
[Creates job that only triggers manually] -> [User does something] -> [Triggers job once]
... sometime later [User does something] -> [ Triggers job once]

Thanks

[agllno]

Sorry for the late reply.
Our use case is as follows:
We use job scheduler to aggregate the historic data and save. This will run for scheduled interval say every hour.
If this scheduled job fails for any reason, we might miss the data as aggregation because raw data will be quarantined. So if we have provision to run one time job for these failed cases it would have been great.

[JackRyanson]

@agllno @dbbaughe sorry for the reply

yes those use cases are correct. Say i have 10M textual document and an NLP service. I want to launch a job that for each document in the 10M indexes performs an NLP analysis and may create more documents in other indexes (E.g. update specific other indexes according to the entities that were found). .. as well as reindex the original document.

This would be a one off, this should start immediately. This should be a job (i can see its status, cancel it etc)

Would this make sense (or instead be something for which you think one should use the basic reindex api in combination with pipeline operators?) do you see yourself adding this capability anytime soon? highly needed in our project here. thanks

[dbbaughe]

Closing in favor of: opensearch-project/job-scheduler#97