Repository navigation
fix: stop after the last block for stop_at_layer=n_layers - #1964
Merged
Merged
Conversation
Blocks stop on entry and nothing follows the last one, so stop_at_layer=n_layers ran to completion and returned logits. It now returns the residual leaving the last block, as HookedTransformer did. Values outside [-n_layers, n_layers] did the same and now raise ValueError, like start_at_layer.
Open
1 task done
7 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
stop_at_layer=n_layersdoesn't stop. Blocks raise the stop on entry and no block follows the last one, so the model runs to completion and returns logits, withln_finalandunembedin the cache. A layer sweep gets ad_vocabtensor on its last step and nothing raises. Out of range values do the same, whilestart_at_layeralready rejects them.stop_at_layer=n_layersnow returns the residual leaving the last block, which is whatHookedTransformerreturned (blocks[:stop_at_layer], thenreturn residual). Anything outside[-n_layers, n_layers]raisesValueError, likestart_at_layer.The stop is a forward hook on the last block module, installed by
forward()for the length of the call. It isn't onhook_outbecause raising in aHookPoint's forward hook loses that point's backward hooks, and OpenAI GPT's block returns a list, which never reacheshook_out. The negative index handling was copied in three places and is now one helper.Type of change
Testing
New tests in
test_bridge_stop_at_layer.pyon the default and compatibility bridges, plus one intest_mamba_adapter.py.dev: 12 of the new tests fail, and the 4 controls pass. With the change:test_bridge_stop_at_layer.py36 passed, andTestMambaStopAtLayerpasses.tests/unit/model_bridge -m "not slow"plus the stop, start andincl_bwdintegration files: 4903 passed, 36 skipped, 3 xfailed.from_configmodels across the adapter factory: all 73 that boot return the residual atstop_at_layer=n_layerson all three entry points. 90 didn't boot this way or were skipped as too large.make check-formatanduv run mypy .are clean.The new
test_openai_gpt_adapter.pytest wasn't run locally because the weights aren't cached (a tiny OpenAI GPT model was checked instead), and the full integration and acceptance tiers are left to CI.Checklist: