Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mitigate overflow when calculating ETA #78

Merged
merged 3 commits into from
Nov 16, 2020

Conversation

csparker247
Copy link
Contributor

Mitigates the overflow that occurs when max progress is very large.

See #77

Comment on lines 2092 to 2094
progress_ > 0 ? static_cast<long long>(elapsed.count() *
(max_progress / progress_))
: 0);
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the single_include is generated, can you make these changes to include/indicators/block_progress_bar.hpp and include/indicators/progress_spinner.hpp?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can do. Is the appropriate call for running the generator documented somewhere so I can make sure single include is updated the correct way as well?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I looked right over it! Thanks!

@csparker247
Copy link
Contributor Author

Unintended consequence of this change is that the time remaining now starts counting up after awhile:

counting-up

Investigating.

@p-ranav p-ranav marked this pull request as draft November 13, 2020 15:14
@csparker247
Copy link
Contributor Author

csparker247 commented Nov 13, 2020

The counting up problem is due to integer truncation. For example, at step 51/100, 100/51 = 1.96 which truncates to 1 for all progress values > 50. This is of course not a problem in the original version because the division happens later.

After thinking about this more, there are three ways we can organize the order of operations here and I think they all have problems:

  • (elapsed * max_prog) / prog: overflows eventually for large values of elapsed and quickly for large values of max_prog
  • elapsed * (max_prog / prog): significant truncation error for normal values of max_prog
  • (elapsed / prog) * max_prog: always 0 for prog > elapsed, which could pretty easily happen if progress is set manually, but probably only for non-default max_prog

It seems like the last option is the least bad, but I only have my use case to go on. An alternative that requires more work but is probably more stable is to keep a rolling average of the iteration rate and use that to project ETA. That appears to be what tqdm does.

EDIT: Or we just do all the math as floats.

@csparker247 csparker247 marked this pull request as ready for review November 16, 2020 16:33
@csparker247
Copy link
Contributor Author

I ended up going for the floating point approach. The std::ceil could probably be removed, but it seemed better to overestimate by a nanosecond than to let it truncate.

@p-ranav p-ranav merged commit 312c49b into p-ranav:master Nov 16, 2020
@csparker247 csparker247 deleted the 77-time-remaining-overflow branch November 16, 2020 21:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants