diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index a9a32a9568e..12c0c0e50ed 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -79,9 +79,6 @@ jobs: - name: "cargo check default features" if: startsWith(matrix.os, 'windows') run: cargo check --all --bins --examples - - run: | - # it should never be a failure not to get the caches, as they can be regenerated. - git lfs fetch && git lfs checkout || true - uses: taiki-e/install-action@v2 with: tool: nextest diff --git a/DEVELOPMENT.md b/DEVELOPMENT.md index 24b5fb3b768..72b2a861ac0 100644 --- a/DEVELOPMENT.md +++ b/DEVELOPMENT.md @@ -1,6 +1,6 @@ # Development Guide -## Practices +## Practices * **test-first development** * protect against regression and make implementing features easy. @@ -9,12 +9,12 @@ for the mundane things, like unhappy code paths. * *use git itself* as reference implementation, and use their test-cases and fixtures where appropriate. At the very least, try to learn from them. - * Run the same test against git whenever feasible to assure git agrees with our implementation. + * Run the same test against git whenever feasible to assure git agrees with our implementation. See `gix-glob` for examples. * *use libgit2* test fixtures and cases where appropriate, or learn from them. * **safety first** * handle all errors, never `unwrap()`. If needed, `expect("why")`. - * provide an error chain and make it easy to understand what went wrong. + * provide an error chain and make it easy to understand what went wrong. * We `thiserror` generally. * Adhere to the [stability guide](https://github.com/Byron/gitoxide/blob/main/STABILITY.md) @@ -28,10 +28,10 @@ The _subject_ usually informs about the *what* and the body provides details and Commit messages _must_ show up in the changelog in case of breaking changes. Examples for that are: - change!: rename `Foo` to `Bar`. (#123) - + And this is why we do it in the body. - remove!: `Repository::obsolete()`. - + Nobody used this method. Features or other changes that are visible and people should know about look like this: @@ -48,7 +48,7 @@ Examples could be: - `make fmt` - thanks clippy -Please refrain from using `chore:` or `refactor:` prefixes as for the most part, users of the API don't care about those. When a `refactor` +Please refrain from using `chore:` or `refactor:` prefixes as for the most part, users of the API don't care about those. When a `refactor` changes the API in some way, prefer to use `feat`, `change`, `rename` or `remove` instead, and most of the time the ones that are not `feat` are breaking so would be seen with their _exclamation mark_ suffix, like `change!`. @@ -63,20 +63,20 @@ Knowing that `cargo smart-release` is driven by commit messages and affects thei to split edits into multiple commits to clearly indicate which crate is actually broken. Typical patterns include making a breaking change in one crate and then fix all others to work with it. For changelogs to look proper -and version bumps to be correct, the first commit would contain only the breaking changes themselves, +and version bumps to be correct, the first commit would contain only the breaking changes themselves, like "rename: `foo()` to `bar()`", and the second commit would contain all changes to adapt to that and look like "adapt to changes in ``". ## Commit History -We generally follow a 'track everything' approach and there is a lot of freedom leading to more commits rather than less. There +We generally follow a 'track everything' approach and there is a lot of freedom leading to more commits rather than less. There is no obligation to squash commits or to otherwise tune the history. We use feature branches and PRs most of the time to be able to take advantage of CI and GitHub review tools, and merge with merge commits -to make individual efforts stand out. There is no need for linearizing history or tuning it in any other way. However, each commit +to make individual efforts stand out. There is no need for linearizing history or tuning it in any other way. However, each commit _must_ follow the guidelines laid out in the `Commit Messages` paragraph. There is value in organizing commits by topic and [_Stacked Git_](https://stacked-git.github.io) is hereby endorsed to do that. - + ## Configuration and overrides As a general rule, respect and implement all applicable [git-config](https://git-scm.com/docs/git-config) by default, but allow the @@ -102,11 +102,11 @@ Parameters which are not available in git or specific to `gitoxide` or the needs * **User Interfaces** * User interfaces can greatly benefit from using async as it's much easier to maintain a responsive UI thread that way thanks to the wonderful future combinators. - * `blocking` can be used to make `Read` and `Iterator` async, or move any operation onto a thread which blends it into the - async world. + * `blocking` can be used to make `Read` and `Iterator` async, or move any operation onto a thread which blends it into the + async world. * Most operations are fast and 'interrupting' them is as easy as ignoring their result by cancelling their task. * Long-running operations can be roughly interacted with using `gix_features::interrupt::trigger()` function, and after a moment - of waiting the flag can be unset with the `…::uninterrupt()` function to allow new long-running operations to work. + of waiting the flag can be unset with the `…::uninterrupt()` function to allow new long-running operations to work. Every long running operation supports this. * **server-side** * ~~Building a pack is CPU and at some point, IO bound, and it makes no sense to use async to handle more connections - git @@ -119,7 +119,7 @@ Parameters which are not available in git or specific to `gitoxide` or the needs * **Why not use it to generate blocking versions of traits automatically?** * This would require `maybe_async` and its dependencies to always be present, increasing compile times. For now we chose a little more code to handle over increasing compile times for everyone. This stance may change later once compile times don't matter that much anymore to allow the removal of code. - + * **`Default` trait implementations** * These can change only if the effect is contained within the callers process. This means **changing the default of a file version** is a **breaking change**. @@ -127,7 +127,7 @@ Parameters which are not available in git or specific to `gitoxide` or the needs * When receiving a `Progress` implementation * without calling `add_child(…)` then use it directly to communicate progress, leaving control of the name to the caller. However, call `.init(…)` to configure the iteration. - * and when calling `add_child(…)` don't use the parent progress instance for anything else. + * and when calling `add_child(…)` don't use the parent progress instance for anything else. * **interruption of long-running operations** * Use `gix-features::interrupt::*` for building support for interruptions of long-running operations only. * It's up to the author to decide how to best integrate it, generally we use a poll-based mechanism to check whether @@ -136,9 +136,9 @@ Parameters which are not available in git or specific to `gitoxide` or the needs * …temporary resources like files might otherwise be leaked. * **this is optional but desirable if…** * …there is no leakage otherwise to support user interfaces. They background long-running operations and need them to be cancellable. - + * **prepare for SHA256 support by using `gix_hash::ObjectId` and `gix_hash::oid`** - * eventually there will be the need to support both Sha1 and Sha256. We anticipate it by using the `Id` type instead + * eventually there will be the need to support both Sha1 and Sha256. We anticipate it by using the `Id` type instead of slices or arrays of 20 bytes. This way, eventually we can support multiple hash digest sizes. * Right now it's unclear how Sha256 is going to work in git, so we only support Sha1 for now. It might be an avenue to proactively implement it ahead of time once there is a specification to follow. @@ -166,7 +166,7 @@ Parameters which are not available in git or specific to `gitoxide` or the needs - Assuming UTF8-ish bytes in paths produced by `git` even on windows due to `MSYS2`, we use `os_str_bytes` to convert these back into `OsStr` and derivatives like `Path` as needed even though it might not always be the case depending on the actual encoding used by `MSYS2` or other abstraction layers, or avoiding to use std types altogether using our own instead. - + ## Sha256 A bunch of notes collected to keep track of what's needed to eventually support it @@ -181,7 +181,7 @@ A bunch of notes collected to keep track of what's needed to eventually support * don't use unwrap, not even in tests. Instead use `quick_error!()` or `Box`. * Use `expect(…)` as assertion on Options, providing context on *why* the expectations should hold. Or in other words, answer "This should work _because_…" - + ## `Options` vs `Context` - Use `Options` whenever there is something to configure in terms of branching behaviour. It can be defaulted, and if it can't these fields should be parameters of the method @@ -195,7 +195,7 @@ In _plumbing_ crates, prefer to default to keeping references if this is feasibl In _porcelain_ crates, like `gix`, we have `Platforms` which are typically cheap enough to create on demand as they configure one or more method calls. These should keep a reference to the `Repository` instance that created them as the user is expected to clone the `Repository` if there is the need. However, if these structures are more expensive, call them `Cache` or `` and prefer to clone the `Repository` into them or otherwise keep them free of lifetimes -to allow the user to keep this structure around for repeated calls. References for this paragraph are [this PR](https://github.com/Canop/bacon/pull/98) and +to allow the user to keep this structure around for repeated calls. References for this paragraph are [this PR](https://github.com/Canop/bacon/pull/98) and [this discussion](https://github.com/Byron/gitoxide/discussions/675). ## Examples, Porcelain CLI and Plumbing CLI - which does what? @@ -240,12 +240,12 @@ by humans. * make it compile quickly, so no extras * **Examples** * An implementation of ideas for actual occasional use and the first step towards getting integrated into Porcelain or Plumbing CLIs. - * Proper command-line parsing with Clap. - * No tests or progress. - * High quality Rust code along with idiomatic `gitoxide` usage so people can learn from it. -* **Plumbing CLI** + * Proper command-line parsing with Clap. + * No tests or progress. + * High quality Rust code along with idiomatic `gitoxide` usage so people can learn from it. +* **Plumbing CLI** * Use Clap AND Argh for command-line parsing via feature toggles to allow for tiny builds as plumbing is mostly for scripts. - * Journey tests + * Journey tests * Progress can be turned on using the `--verbose` flag, quiet by default. * Examples can be turned into plumbing by adding journey tests and `argh` command-line parsing, as well as progress. * **Porcelain CLI** @@ -262,9 +262,8 @@ get an overview. ## Reviewing PRs -- be sure to clone locally and run tests with `GIX_TEST_IGNORE_ARCHIVES=1` to assure new fixture scripts (if there are any) are validated - on _MacOS_ and _Windows_. Note that linux doesn't need to be tested that way as CI on linux ignores them by merit of not checking them out - via `gix-lfs`. +- Be sure to clone locally and run tests with `GIX_TEST_IGNORE_ARCHIVES=1` to assure new fixture scripts (if there are any) are validated + on _MacOS_ and _Windows_. Linux doesn't need to be tested locally that way, as CI on Linux includes it. ## Creating a release @@ -309,5 +308,5 @@ GIT_SSH_COMMAND="ssh -VVV" \ git ``` -Consider adding `GIT_TRACE2_PERF=1` (possibly add `GIT_TRACE2_PERF_BRIEF=1` for brevity) as well for statistics and variables +Consider adding `GIT_TRACE2_PERF=1` (possibly add `GIT_TRACE2_PERF_BRIEF=1` for brevity) as well for statistics and variables (see [their source for more](https://github.com/git/git/blob/b50a608ba20348cb3dfc16a696816d51780e3f0f/trace2/tr2_sysenv.c#L50). diff --git a/README.md b/README.md index ee59a75d33a..a52a4c841c5 100644 --- a/README.md +++ b/README.md @@ -3,10 +3,10 @@ `gitoxide` is an implementation of `git` written in Rust for developing future-proof applications which strive for correctness and -performance while providing a pleasant and unsurprising developer experience. +performance while providing a pleasant and unsurprising developer experience. `gitoxide` provides the `gix` and `ein` binaries for use on the command-line to allow experimentation with key features -like `fetch` and `clone`, and to validate the usability and control of the API offered by the [`gix`] crate. +like `fetch` and `clone`, and to validate the usability and control of the API offered by the [`gix`] crate. `gitoxide` aspires to be a production-grade server implementation and the `ein` binary aspires to become the default way to interact with git repositories. @@ -16,10 +16,10 @@ like `fetch` and `clone`, and to validate the usability and control of the API o ## Development Status -The command-line tools as well as the status of each crate is described in +The command-line tools as well as the status of each crate is described in [the crate status document](https://github.com/Byron/gitoxide/blob/main/crate-status.md). -For use in applications, look for the [`gix`](https://github.com/Byron/gitoxide/blob/main/crate-status.md#gix) crate, +For use in applications, look for the [`gix`](https://github.com/Byron/gitoxide/blob/main/crate-status.md#gix) crate, which serves as entrypoint to the functionality provided by various lower-level plumbing crates like [`gix-config`](https://github.com/Byron/gitoxide/blob/main/crate-status.md#gix-config). @@ -66,10 +66,10 @@ Follow linked crate name for detailed status. Please note that all crates follow * **Stability Tier 1** - [gix-lock](https://github.com/Byron/gitoxide/blob/main/gix-lock/README.md) - + * **Stability Tier 2** - [gix-tempfile](https://github.com/Byron/gitoxide/blob/main/gix-tempfile/README.md) - + ### Stabilization Candidates Crates that seem feature complete and need to see some more use before they can be released as 1.0. @@ -144,13 +144,13 @@ is usable to some extent. * [gix-tix](https://github.com/Byron/gitoxide/blob/main/crate-status.md#gix-tix) * [gix-bundle](https://github.com/Byron/gitoxide/blob/main/crate-status.md#gix-bundle) * [gix-fsck](https://github.com/Byron/gitoxide/blob/main/crate-status.md#gix-fsck) - + ### Stress Testing * [x] Verify huge packs - * [x] Explode a pack to disk + * [x] Explode a pack to disk * [x] Generate and verify large commit graphs * [ ] Generate huge pack from a lot of loose objects - + ### Stability and MSRV Our [stability guide] helps to judge how much churn can be expected when depending on crates in this workspace. @@ -161,7 +161,7 @@ Our [stability guide] helps to judge how much churn can be expected when dependi ### Download a Binary Release -Using `cargo binstall`, one is able to fetch [binary releases][releases]. You can install it via `cargo install cargo-binstall`, assuming +Using `cargo binstall`, one is able to fetch [binary releases][releases]. You can install it via `cargo install cargo-binstall`, assuming the [rust toolchain][rustup] is present. Then install gitoxide with `cargo binstall gitoxide`. @@ -200,7 +200,7 @@ for packagers who need to tune external dependencies. ``` # A certain way to install `gitoxide` with just Rust and a C compiler installed. # If there are problems with SSL certificates during clones, try to omit `--locked`. -cargo install gitoxide --locked --no-default-features --features max-pure +cargo install gitoxide --locked --no-default-features --features max-pure # The default installation, 'max', is the fastest, but also needs some libraries available to build successfully. # Installing these is platform-dependent and thus can't be explained here. @@ -227,7 +227,7 @@ What follows is a list of known failures. - On Fedora, `perl` needs to be installed for `OpenSSL` to build properly. This can be done with the following command: `dnf install perl` (see [this issue](https://github.com/Byron/gitoxide/issues/592)). -[releases]: https://github.com/Byron/gitoxide/releases +[releases]: https://github.com/Byron/gitoxide/releases [rustup]: https://rustup.rs ## Usage @@ -269,7 +269,7 @@ Project goals can change over time as we learn more, and they can be challenged. * With the tools and experience available here there is no reason not to support Windows. * [Windows is tested on CI](https://github.com/Byron/gitoxide/blob/df66d74aa2a8cb62d8a03383135f08c8e8c579a8/.github/workflows/rust.yml#L34) and failures do prevent releases. - + ## Non-Goals Project non-goals can change over time as we learn more, and they can be challenged. @@ -282,7 +282,7 @@ Project non-goals can change over time as we learn more, and they can be challen * **use async IO everywhere** * for the most part, git operations are heavily reliant on memory mapped IO as well as CPU to decompress data, which doesn't lend itself well to async IO out of the box. - * Use `blocking` as well as `gix-features::interrupt` to bring operations into the async world and to control + * Use `blocking` as well as `gix-features::interrupt` to bring operations into the async world and to control long running operations. * When connecting or streaming over TCP connections, especially when receiving on the server, async seems like a must though, but behind a feature flag. @@ -291,12 +291,9 @@ Project non-goals can change over time as we learn more, and they can be challen If what you have seen so far sparked your interest to contribute, then let us say: We are happy to have you and help you to get started. -> ❗️Note❗️: For cloning, `gix-lfs` needs to be locally installed or the checkout will fail. `git lfs install` must have been called once, followed - by `git lfs pull` to replace the `lfs`-pointer files. - We recommend running `just test check-size` during the development process to assure CI is green before pushing. -A backlog for work ready to be picked up is [available in the Project's Kanban board][project-board], which contains instructions on how +A backlog for work ready to be picked up is [available in the Project's Kanban board][project-board], which contains instructions on how to pick a task. If it's empty or you have other questions, feel free to [start a discussion][discussions] or reach out to @Byron [privately][keybase]. For additional details, also take a look at the [collaboration guide]. @@ -314,7 +311,7 @@ For additional details, also take a look at the [collaboration guide]. - Get an introduction to `gitoxide` itself which should be a good foundation for any contribution, but isn't a requirement for contributions either. - [Gifting Gitoxide](https://www.youtube.com/playlist?list=PLMHbQxe1e9MlhyyZQXPi_dc-bKudE-WUw) - See how PRs are reviewed along with a lot of inner monologue. - + #### Other Media - [Rustacean Station Podcast](https://rustacean-station.org/episode/055-sebastian-thiel/) @@ -376,20 +373,20 @@ Provide a CLI to for the most basic user journey: ## Shortcomings & Limitations Please take a look at the [`SHORTCOMINGS.md` file](https://github.com/Byron/gitoxide/blob/main/SHORTCOMINGS.md) for details. - + ## Credits * **itertools** _(MIT Licensed)_ * We use the `izip!` macro in code * **deflate2** _(MIT Licensed)_ * We use various abstractions to implement decompression and compression directly on top of the rather low-level `miniz_oxide` crate - + ## 🙏 Special Thanks 🙏 At least for now this section is exclusive to highlight the incredible support that [Josh Triplett](https://github.com/joshtriplett) has provided to me in the form of advice, sponsorship and countless other benefits that were incredibly meaningful. Going full time with `gitoxide` would hardly have been feasible without his involvement, and I couldn't be more grateful 😌. - + ## License This project is licensed under either of @@ -405,6 +402,6 @@ at your option. * Originally @Byron was really fascinated by [this problem](https://github.com/gitpython-developers/GitPython/issues/765#issuecomment-396072153) and believes that with `gitoxide` it will be possible to provide the fastest solution for it. -* @Byron has been absolutely blown away by `git` from the first time he experienced git more than 13 years ago, and +* @Byron has been absolutely blown away by `git` from the first time he experienced git more than 13 years ago, and tried to implement it in [various shapes](https://github.com/gitpython-developers/GitPython/pull/1028) and [forms](https://github.com/byron/gogit) multiple [times](https://github.com/Byron/gitplusplus). Now with Rust @Byron finally feels to have found the right tool for the job! diff --git a/tests/tools/src/lib.rs b/tests/tools/src/lib.rs index 145e24ecd23..a55afde5816 100644 --- a/tests/tools/src/lib.rs +++ b/tests/tools/src/lib.rs @@ -29,7 +29,7 @@ pub use tempfile; /// /// #[test] /// fn this() -> Result { -/// let x: usize = "42".parse()?; +/// let x: usize = "42".parse()?; /// Ok(()) /// /// } @@ -264,8 +264,8 @@ fn fixture_bytes_inner(path: impl AsRef, root: DirectoryRoot) -> Vec { /// If a script result doesn't exist, these will be checked first and extracted if present, which they are by default. /// This behaviour can be prohibited by setting the `GIX_TEST_IGNORE_ARCHIVES` to any value. /// -/// To speed CI up, one can add these archives to the repository. It's absolutely recommended to use `gix-lfs` for that to -/// not bloat the repository size. +/// To speed CI up, one can add these archives to the repository. Since LFS is not currently being used, it is +/// important to check their size first, though in most cases generated archives will not be very large. /// /// #### Disable Archive Creation ///