{
  "version": "1.0",
  "title": "Siddhant Rathi — The Stream",
  "description": "Chronological research logs, model checkpoints, raw curiosities, and working prototypes.",
  "author": {
    "name": "Siddhant Rathi",
    "handle": "siddhant-rathi",
    "title": "AI Systems Engineer"
  },
  "entries": [
    {
      "id": "entry-01",
      "timestamp": "2026.04.12 03:14",
      "category": "Engineering",
      "title": "Sparse Autoencoders and the geometry of conceptual superposition",
      "summary": "Dissecting layers 18–24: when projecting activations into sparse dictionary bases, superposition resolves into clean monosemantic axes.",
      "content": "Many unique and strange things about representations in deep networks — their polysemanticity, their geometry, their capacity for superposition — trace back to high-dimensional projections. Dissecting layers 18–24: when projecting activations into sparse dictionary bases (expansion factor `k=32`), the superposition resolves into clean monosemantic axes corresponding to abstract systemic failure, irony, and temporal recursion.\n\n```python:sae_geometry.py\n# Extract top-k latent projections\ndef project_sparse(x, W_enc, k=32):\n    pre_act = x @ W_enc.t() + b_enc\n    topk_v, topk_idx = torch.topk(pre_act, k)\n    sparse_repr = torch.zeros_like(pre_act)\n    return sparse_repr.scatter_(-1, topk_idx, topk_v)\n```",
      "tags": ["SAE", "Mechanistic Interpretability", "PyTorch"],
      "link": {
        "text": "View mechanistic analysis notes",
        "url": "#"
      }
    },
    {
      "id": "entry-02",
      "timestamp": "2025.11.02 09:03",
      "category": "Engineering",
      "title": "Continuous Generative Trajectories & Latent ODEs",
      "summary": "Steering continuous generative trajectories in real time via low-rank cross-attention perturbations.",
      "content": "Steering continuous generative trajectories in real time via low-rank cross-attention perturbations. Rather than discrete prompt token steps, user drag shifts dynamic equilibrium across the manifold. Jazz bands don't succeed by being paid the most or doing the most gigs; they succeed by making the best art. If we can build models that teach us something new about latent space every day, that is where I want to be.",
      "tags": ["Diffusion", "Euler-ODE", "Interaction"],
      "link": {
        "text": "Latent Canvas interactive demonstration (Euler-ODE Stepper) →",
        "url": "#"
      }
    },
    {
      "id": "entry-03",
      "timestamp": "2025.09.29 01:45",
      "category": "Reading",
      "title": "Proving Yourself",
      "summary": "Interior reflections on operating from abundance vs scarcity in creative and technical lifework.",
      "content": "My frequent interlocutors will know that I've been captivated recently by the idea of proving oneself, and the process by which people I admire have crossed over from a stage of life where they were 'trying to prove themselves' to one in which they operate from a place of abundance about their capabilities and the worth of their contributions to the world.\n\nI find that people who have proven themselves, creating and operating from a place of abundance, find it natural to do their lifework. Accordingly, it's in the interest of our civilization to find a way to help as many people cross the chasm, as early on in their careers as possible.\n\n> 1. Proving yourself is mostly an interior matter. You prove yourself *to yourself*. This is the only way to cross the chasm.\n>\n> 2. You prove yourself when you begin creating and operating in the world from a feeling of abundance about your worth and capabilities, rather than from a place of scarcity.\n>\n> 3. In people who have proven themselves, their inner voice prevails more often than the voices around them when they are in disagreement.",
      "tags": ["Philosophy", "Essays", "Mindset"]
    },
    {
      "id": "entry-04",
      "timestamp": "2025.09.28 22:52",
      "category": "Reading",
      "title": "On Basins of Attraction & Tacit Knowledge",
      "summary": "Reflections on Michael Polanyi and emergent optimization in cognitive systems.",
      "content": "> “We do not program intelligence; we cultivate basins of attraction and listen for resonance.”\n\n— Fragment #89 on emergent optimization, cross-referenced with W. Ross Ashby's *Design for a Brain* and Michael Polanyi's *The Tacit Dimension*.\n\nWhen we calibrate loss surfaces, we are not issuing procedural directives. We sculpt topological valleys where coherence is the lowest-energy configuration.",
      "tags": ["Cybernetics", "Polanyi", "Book Notes"]
    },
    {
      "id": "entry-05",
      "timestamp": "2025.08.14 18:40",
      "category": "Life",
      "title": "Ryoan-ji: Gravel Rhythms and Tactile Stillness",
      "summary": "Photographic study of negative space and stone alignments at Ryoan-ji in Kyoto.",
      "content": "Spent an hour at dawn observing the 15 stones of Ryoan-ji. From whatever angle you view the courtyard, at least one stone remains hidden from sight. It is a physical manifestation of incomplete observation in high dimensions: you cannot collapse the whole system from a single vantage point.\n\n![Archival Plate 04 — Kyoto dry landscape garden (Karesansui). Nikon Zf, 40mm f/2.](assets/kyoto-garden.jpg)",
      "tags": ["Kyoto", "Photography", "Stillness"]
    },
    {
      "id": "entry-06",
      "timestamp": "2025.03.28 11:05",
      "category": "Engineering",
      "title": "Run #409: 7B Mixture-of-Depths convergence at 1.4T tokens",
      "summary": "Compute-optimal dynamic routing verification reducing active parameters from 6.8B to 3.2B.",
      "content": "Completed compute-optimal dynamic routing verification. Preserved 98.4% validation cross-entropy while reducing active parameters per token from 6.8B to 3.2B. FLOPs efficiency jumped 2.14x (+114%) on A100-80G FP16 cluster.\n\n```bash:telemetry\nWeights: hf://siddhant-rathi/mod-7b-v409\nCheckpoint: step_700000_loss_1.682\nSha256: 7e93f810cb92aa4e9314810da567a1c04\n```",
      "tags": ["MoD", "Distributed", "Inference"]
    }
  ]
}