Skip to content

Change polling for progress logging to exponential backoff #873

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

FlorentinD
Copy link
Contributor

No description provided.

Copy link

netlify bot commented Apr 25, 2025

Deploy Preview for neo4j-graph-data-science-client canceled.

Name Link
🔨 Latest commit 1a40613
🔍 Latest deploy log https://app.netlify.com/sites/neo4j-graph-data-science-client/deploys/680ba47d952b0500092729bd

@FlorentinD FlorentinD requested a review from Copilot April 25, 2025 15:10
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR changes the polling mechanism for progress logging to use exponential backoff, reducing the number of calls to check for progress updates while upgrading the Neo4j image version in the test environment.

  • Update Neo4j image version in compose.yml
  • Introduce exponential backoff using tenacity’s wait_exponential in several modules
  • Add a test for validating the new logging interval behavior

Reviewed Changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated no comments.

Show a summary per file
File Description
scripts/test_envs/gds_plugin_enterprise/compose.yml Updated Neo4j image to neo4j:5-enterprise
graphdatascience/tests/unit/query_runner/progress/test_query_progress_logger.py Added test_log_interval verifying progress logging interval
graphdatascience/retry_utils/retry_utils.py Introduced new retry_until_future class for async retry based on a Future
graphdatascience/query_runner/session_query_runner.py Updated progress logger to use wait_exponential for polling
graphdatascience/query_runner/progress/query_progress_logger.py Switched polling interval to exponential backoff and adjusted progress bar parameters
graphdatascience/query_runner/neo4j_query_runner.py Updated progress logger instantiation to use exponential backoff
changelog.md Updated changelog reflecting reduced progress update calls
Comments suppressed due to low confidence (3)

graphdatascience/retry_utils/retry_utils.py:24

  • [nitpick] The class name 'retry_until_future' does not follow the standard PascalCase naming convention; consider renaming it to 'RetryUntilFuture'.
class retry_until_future(retry_base):

graphdatascience/query_runner/progress/query_progress_logger.py:40

  • [nitpick] The error message refers to 'polling interval' while the parameter is named 'log_interval'; consider updating the error message for clarity.
raise ValueError("polling interval must be a float or an instance of wait_base")

graphdatascience/tests/unit/query_runner/progress/test_query_progress_logger.py:43

  • The use of time.sleep(0.5) requires importing the time module. Please add 'import time' at the top of the file.
time.sleep(0.5)

future = executor.submit(runnable)

futures.wait([future], timeout=self._initial_wait_time) # wait for progress task to be available
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should run the progress logger also inside the threadpool.

Check every 0.5s if the main query still runs + cancel progress thread if its done.
right now we might delay the execution until 10s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant