Tensor-Slayer: LLM model editing directly on the weights, without fine-tuning

Portfolio · by Shreenivas Sangoram · updated

Short answer

Tensor-Slayer is an MIT-licensed tool for exploring and editing a language model at the tensor level. An agent analyses tensors (statistics, distributions, patterns), recommends targeted edits, and applies them to the weights file with before-and-after previews, safety checks and automatic backups, with no fine-tuning or dataset. It started a line of work that runs through rank-one fact edits and layer-stacked merges.

Fine-tuning changes a model by showing it data. Tensor-Slayer asks a different question: how far can you get by reading the weights and editing them directly?

What the tool does

  • Explore a model's structure, with a focus on attention, MLP and embedding tensors.
  • Analyse individual tensors: minimum, maximum, mean, standard deviation, sparsity, percentile distributions, pattern detection and simulated operations.
  • Patch with guard rails: choose an operation (scale, add, normalise), a target and a value, preview the before and after statistics, pass safety checks, and keep an automatic backup.
  • Adapt edits to position: layer-aware modifications based on where a tensor sits and what role it plays.
  • Ask an agent for recommendations aimed at a capability such as maths or reasoning.

The repository includes the patches and the evaluation results it reports on Qwen 0.6B (MMLU and HumanEval). Those are the repository's own measurements; treat them as a starting point, not a benchmark claim.

The line of work

  1. Editing quantised GGUF weights in a UI; a C runtime for running Gemma on a laptop GPU
  2. Frankenmerge: layer-stacked Qwen and GLM merges, healed with SFT and RL
  3. Rank-one fact edits on Qwen3 (Qwen-ROME, public)
  4. Surgical Theatre: a zero-copy context manager for temporary weight edits during RL (public)
  5. Tensor-Slayer 0.2: DSPy generates the patches, nothing hard-coded
  6. Autonomous pruning: a DSPy agent chooses the structured-pruning strategy, processing layer by layer for models larger than RAM
  7. Inferenceless interpretability: point at a local checkpoint, explore its tensors and discuss the architecture with an assisting LLM
  8. Statistical tensor analysis and merging across incompatible architectures
  9. Tensor-Slayer released: an agent proposes targeted tensor operations
  10. Hex-patching QwQ-32B: a tensor explorer and forward-pass tracer

Facts

ItemDetail
Repositoryareu01or00/Tensor-Slayer, MIT licence
FormatWorks on safetensors files directly
Related public workQwen-ROME, tensorslayereval

What you can buy

Environments where an agent has to change a model's behaviour under constraints: edit or merge weights, keep a frozen evaluation from regressing, and stay within a compute budget. Graders come from the evaluation set and the constraints.

FAQ

Can you improve a model without fine-tuning?

Sometimes, in narrow ways. Targeted edits to specific tensors, rank-one fact edits and merges can change behaviour without a training run. Whether an edit helps has to be measured on a held-out evaluation, every time.

What is ROME model editing?

Rank-One Model Editing changes a single stored fact by applying a rank-one update to one MLP layer's weights. Qwen-ROME applies it to Qwen3 models.

Is Tensor-Slayer open source?

Yes, under the MIT licence.

Want this, or something like it?

License it, or have me build an environment in the same style.

Email me →