1# Caching in the new trait solver23Caching results of the trait solver is necessary for performance.4We have to make sure that it is sound. Caching is handled by the5[`SearchGraph`]67[`SearchGraph`]: https://github.com/rust-lang/rust/blob/7606c13961ddc1174b70638e934df0439b7dc515/compiler/rustc_trait_selection/src/solve/search_graph.rs#L102-L11789## The global cache1011At its core, the cache is fairly straightforward. When evaluating a goal, we12check whether it's in the global cache. If so, we reuse that entry. If not, we13compute the goal and then store its result in the cache.1415To handle incremental compilation the computation of a goal happens inside of16[`DepGraph::with_anon_task`][`with_anon_task`] which creates a new `DepNode` which depends on all queries17used inside of this computation. When accessing the global cache we then read this18`DepNode`, manually adding a dependency edge to all the queries used: [source][wdn].1920### Dealing with overflow2122Hitting the recursion limit is not fatal in the new trait solver but instead simply23causes it to return ambiguity: [source][overflow]. Whether we hit the recursion limit24can therefore change the result without resulting in a compilation failure. This25means we must consider the remaining available depth when accessing a cache result.2627We do this by storing more information in the cache entry. For goals whose evaluation28did not reach the recursion limit, we simply store its reached depth: [source][req-depth].29These results can freely be used as long as the current `available_depth` is higher than30its `reached_depth`: [source][req-depth-ck]. We then update the reached depth of the31current goal to make sure that whether we've used the global cache entry is not32observable: [source][update-depth].3334For goals which reach the recursion limit we currently only use the cached result if the35available depth *exactly matches* the depth of the entry. The cache entry for each goal36therefore contains a separate result for each remaining depth: [source][rem-depth].[^1]3738## Handling cycles3940The trait solver has to support cycles. These cycles are either inductive or coinductive,41depending on the participating goals. See the [chapter on coinduction] for more details.42We distinguish between the cycle heads and the cycle root: a stack entry is a43cycle head if it recursively accessed. The *root* is the deepest goal on the stack which44is involved in any cycle. Given the following dependency tree, `A` and `B` are both cycle45heads, while only `A` is a root.4647```mermaid48graph TB49    A --> B50    B --> C51    C --> B52    C --> A53```5455The result of cycle participants depends on the result of goals still on the stack.56However, we are currently computing that result, so its result is still unknown. This is57handled by evaluating cycle heads until we reach a fixpoint. In the first iteration, we58return either success or overflow with no constraints, depending on whether the cycle is59coinductive: [source][initial-prov-result]. After evaluating the head of a cycle, we60check whether its [`provisional_result`] is equal to the result of this iteration. If so,61we've finished evaluating this cycle and return its result. If not, we update the provisional62result and reevaluate the goal: [source][fixpoint]. After the first iteration it does not63matter whether cycles are coinductive or inductive. We always use the provisional result.6465### Only caching cycle roots6667We cannot move the result of any cycle participant to the global cache until we've68finished evaluating the cycle root. However, even after we've completely evaluated the69cycle, we are still forced to discard the result of all participants apart from the root70itself.7172We track the query dependencies of all global cache entries. This causes the caching of73cycle participants to be non-trivial. We cannot simply reuse the `DepNode` of the cycle74root.[^2] If we have a cycle `A -> B -> A`, then the `DepNode` for `A` contains a dependency75from `A -> B`. Reusing this entry for `B` may break if the source is changed. The `B -> A`76edge may not exist anymore and `A` may have been completely removed. This can easily result77in an ICE.7879However, it's even worse as the result of a cycle can change depending on which goal is80the root: [example][unstable-result-ex]. This forces us to weaken caching even further.81We must not use a cache entry of a cycle root, if there exists a stack entry, which was82a participant of its cycle involving that root. We do this by storing all cycle participants83of a given root in its global cache entry and checking that it contains no element of the84stack: [source][cycle-participants].8586### The provisional cache8788TODO: write this :38990- stack dependence of provisional results91- edge case: provisional cache impacts behavior929394[`with_anon_task`]: https://github.com/rust-lang/rust/blob/7606c13961ddc1174b70638e934df0439b7dc515/compiler/rustc_trait_selection/src/solve/search_graph.rs#L39195[wdn]: https://github.com/rust-lang/rust/blob/7606c13961ddc1174b70638e934df0439b7dc515/compiler/rustc_middle/src/traits/solve/cache.rs#L7896[overflow]: https://github.com/rust-lang/rust/blob/7606c13961ddc1174b70638e934df0439b7dc515/compiler/rustc_trait_selection/src/solve/search_graph.rs#L27697[req-depth]: https://github.com/rust-lang/rust/blob/7606c13961ddc1174b70638e934df0439b7dc515/compiler/rustc_middle/src/traits/solve/cache.rs#L10298[req-depth-ck]: https://github.com/rust-lang/rust/blob/7606c13961ddc1174b70638e934df0439b7dc515/compiler/rustc_middle/src/traits/solve/cache.rs#L76-L8699[update-depth]: https://github.com/rust-lang/rust/blob/7606c13961ddc1174b70638e934df0439b7dc515/compiler/rustc_trait_selection/src/solve/search_graph.rs#L308100[rem-depth]: https://github.com/rust-lang/rust/blob/7606c13961ddc1174b70638e934df0439b7dc515/compiler/rustc_middle/src/traits/solve/cache.rs#L124101[^1]: This is overly restrictive: if all nested goals return the overflow response with some102available depth `n`, then their result should be the same for any depths smaller than `n`.103We can implement this optimization in the future.104105[chapter on coinduction]: ./coinduction.md106[`provisional_result`]: https://github.com/rust-lang/rust/blob/7606c13961ddc1174b70638e934df0439b7dc515/compiler/rustc_trait_selection/src/solve/search_graph.rs#L57107[initial-prov-result]: https://github.com/rust-lang/rust/blob/7606c13961ddc1174b70638e934df0439b7dc515/compiler/rustc_trait_selection/src/solve/search_graph.rs#L366-L370108[fixpoint]: https://github.com/rust-lang/rust/blob/7606c13961ddc1174b70638e934df0439b7dc515/compiler/rustc_trait_selection/src/solve/search_graph.rs#L425-L446109[^2]: summarizing the relevant [Zulip thread]110111[zulip thread]: https://rust-lang.zulipchat.com/#narrow/stream/364551-t-types.2Ftrait-system-refactor/topic/global.20cache112[unstable-result-ex]: https://github.com/rust-lang/rust/blob/7606c13961ddc1174b70638e934df0439b7dc515/tests/ui/traits/next-solver/cycles/coinduction/incompleteness-unstable-result.rs#L4-L16113[cycle-participants]: https://github.com/rust-lang/rust/blob/7606c13961ddc1174b70638e934df0439b7dc515/compiler/rustc_middle/src/traits/solve/cache.rs#L72-L74