-
Notifications
You must be signed in to change notification settings - Fork 52
Change polling for progress logging to exponential backoff #873
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
✅ Deploy Preview for neo4j-graph-data-science-client canceled.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR changes the polling mechanism for progress logging to use exponential backoff, reducing the number of calls to check for progress updates while upgrading the Neo4j image version in the test environment.
- Update Neo4j image version in compose.yml
- Introduce exponential backoff using tenacity’s wait_exponential in several modules
- Add a test for validating the new logging interval behavior
Reviewed Changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated no comments.
Show a summary per file
File | Description |
---|---|
scripts/test_envs/gds_plugin_enterprise/compose.yml | Updated Neo4j image to neo4j:5-enterprise |
graphdatascience/tests/unit/query_runner/progress/test_query_progress_logger.py | Added test_log_interval verifying progress logging interval |
graphdatascience/retry_utils/retry_utils.py | Introduced new retry_until_future class for async retry based on a Future |
graphdatascience/query_runner/session_query_runner.py | Updated progress logger to use wait_exponential for polling |
graphdatascience/query_runner/progress/query_progress_logger.py | Switched polling interval to exponential backoff and adjusted progress bar parameters |
graphdatascience/query_runner/neo4j_query_runner.py | Updated progress logger instantiation to use exponential backoff |
changelog.md | Updated changelog reflecting reduced progress update calls |
Comments suppressed due to low confidence (3)
graphdatascience/retry_utils/retry_utils.py:24
- [nitpick] The class name 'retry_until_future' does not follow the standard PascalCase naming convention; consider renaming it to 'RetryUntilFuture'.
class retry_until_future(retry_base):
graphdatascience/query_runner/progress/query_progress_logger.py:40
- [nitpick] The error message refers to 'polling interval' while the parameter is named 'log_interval'; consider updating the error message for clarity.
raise ValueError("polling interval must be a float or an instance of wait_base")
graphdatascience/tests/unit/query_runner/progress/test_query_progress_logger.py:43
- The use of time.sleep(0.5) requires importing the time module. Please add 'import time' at the top of the file.
time.sleep(0.5)
future = executor.submit(runnable) | ||
|
||
futures.wait([future], timeout=self._initial_wait_time) # wait for progress task to be available |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we should run the progress logger also inside the threadpool.
Check every 0.5s if the main query still runs + cancel progress thread if its done.
right now we might delay the execution until 10s
No description provided.