fix(ruby): a call on an instance variable links the class initialize gives it

CHANGELOG.md

@@ -199,6 +199,7 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 - In C++ and Rust, a call written with a qualifier, like `Status::OK()`, now links to a method of the class it names, never to one of a class whose name only ends the same way. Before, facebook/rocksdb's calls to `Status::OK()`, `Status::InvalidArgument(…)` and `Status::Corruption(…)` linked to `IOStatus`'s methods of those names, a subclass calling its base, like `return FileSystemWrapper::NewWritableFile(…)` or google/leveldb's `return DB::Put(…)`, linked to another subclass or to the calling method itself, and BurntSushi/ripgrep's `Path::new(…)` on the standard library's `Path` linked to its own `PrinterPath::new`. When classes in several languages share the name, the one in the call's own language is taken, so protocolbuffers/protobuf's C++ `DescriptorPool::generated_pool()` calls, which met its Ruby namesake first and linked to nothing, now link to the C++ method. A C++ call through a template parameter, like `CacheShard::ComputeHash(key)` in rocksdb's `template <class CacheShard> class ShardedCache`, no longer links to a class that merely shares words with the parameter's name, unless that class is the parameter's default, like `TcParser` in `template <typename Parser = TcParser>`. A template's value parameter, like `const PrefetchOpts& kOpts`, is no longer mistaken for a type parameter, so references to `PrefetchOpts` inside it now link. Re-index C++ and Rust projects after upgrading.
 - In Ruby, a call on a class written with its namespace, like `Billing::ChargeCard.call(…)` or `::Billing::ChargeCard.call(…)`, now links the caller to that class and to the class method it calls, as `ChargeCard.call(…)` already linked to the class. Before, a class used only through such calls looked like it had no callers. The name is looked up the way Ruby does it, from the namespaces around the call outward, so `Users::Create.call` reaches `Users::Create` and never a `Billing::Create`, a call inside `module Api; module V2` reaches `Api::V2`'s class before another version's, and a library's class like `ActiveRecord::Base` is never taken for a project class of the same last name. Re-index Ruby projects after upgrading. Thanks @james-jang for the report and @drakeo338 for the pull request. (#2503)
 - In Ruby, a call to a method whose name ends in `!` or `?` now links when its receiver's class is known, like `writer.flush!` or `writer.ready?` after `writer = CacheWriter.new`, or `CacheWriter.create!(…)` on the class. A method called straight on a new object, like `CacheWriter.new(key).write_cache` or `Billing::Ledger.new.charge!`, now links to that class's method or to one it inherits. Before, all of these linked nothing. The class is looked up the way Ruby does it, from the namespaces around the call outward, and one from outside your project, like rack's `Request` used inside `Rack::Protection`, links nothing rather than a project class of the same name. Re-index Ruby projects after upgrading. Thanks @ricobl.
+- In Ruby, a call on an instance variable, like `@writer.flush`, now links to the method of the class the variable holds when the class's `initialize` sets it to a new object of one class, like `@writer = CacheWriter.new`. Before, no call on an instance variable linked. A variable that `initialize` sets to anything else, or to objects of two classes, stays unlinked rather than guessed. Re-index Ruby projects after upgrading.
 - In PHP, a class written through a namespace alias in a different case than its `use` statement, like `new field\FirstName()` after `use App\Fields as Field;`, now links to that class, since PHP reads namespace names without regard to case. Before, the name was read relative to the file's own namespace and stayed unlinked. Re-index PHP projects after upgrading. Thanks @danusha2345. (#2369)
 - In C++, a call through a class template's specialization written with its arguments, like fmtlib/fmt's `formatter<int>::format(42, ctx)`, no longer links to a member defined for another specialization, like a test's `formatter<incomplete_type>::format`, and when the template declares no such member, as for `formatter<std::tm, Char>::parse(ctx)`, the call is no longer matched to another class's method of that name, like `native_formatter::parse`. A call to the specialization a member is defined for, and to a template's own members, links as before. Re-index C++ projects after upgrading.
 - In C++, a function is now named by its declarator, never by something written in the clauses after its parameters. nlohmann/json's constructor `basic_json(CompatibleType&& val) noexcept(noexcept(JSONSerializer<U>::to_json(…)))` was missing from the index, replaced by a method `JSONSerializer::to_json` that doesn't exist and that other `JSONSerializer<T>::to_json(…)` calls linked to; a method declared with `throw(std::bad_alloc)` was indexed as `std::bad_alloc`, and a function with a trailing `requires std::integral<T>` as `std::integral<T>`. Re-index C++ projects after upgrading.

__tests__/ruby-ivar-receivers.test.ts

@@ -0,0 +1,167 @@
+/**
+ * A Ruby call on an instance variable, like `@writer.flush`, links to the
+ * method of the class the class's `initialize` gives the variable:
+ * `@writer = CacheWriter.new` makes it a `CacheWriter`. Before, no call on an
+ * instance variable linked at all.
+ *
+ * Only `initialize` is read, and only a lone `Const.new(…)` types the
+ * variable there: one assigned two classes, a conditional, a value passed
+ * through `.tap`, or anything but `.new` leaves it untyped, and a class the
+ * project doesn't declare (`Logger.new($stdout)`) links nothing. A class
+ * method's `@x` is the class's own, and a module's methods run on whatever
+ * includes it, so neither is typed.
+ *
+ * Runs on the native kernel (skipped when none is built) and the wasm
+ * extractor.
+ */
+import { describe, it, expect, afterEach } from 'vitest';
+import * as fs from 'fs';
+import * as os from 'os';
+import * as path from 'path';
+import { CodeGraph } from '../src';
+import { getKernel } from '../src/extraction/kernel/loader';
+
+const FILES: Record<string, string> = {
+  'app/writer.rb': `class CacheWriter
+  def flush
+    nil
+  end
+
+  def ready?
+    true
+  end
+end
+
+class OtherWriter
+  def flush
+    nil
+  end
+end
+
+module Billing
+  class Card
+    def charge!(amount)
+      amount
+    end
+  end
+
+  class Ledger < Card
+  end
+end
+`,
+  'app/holder.rb': `class Holder
+  def initialize(key)
+    @writer = CacheWriter.new
+    @lazy ||= OtherWriter.new(
+      key
+    )
+    @either = key ? CacheWriter.new : OtherWriter.new
+    @twice = CacheWriter.new
+    @twice = OtherWriter.new if key
+    @ledger = Billing::Ledger.new key do |l|
+      l
+    end
+    @tapped = CacheWriter.new.tap { |w| w }
+    @outside = Logger.new($stdout)
+    @a, @b = CacheWriter.new, OtherWriter.new
+  end
+
+  def via_ivar
+    @writer.flush
+    @writer.ready?
+    @lazy.flush
+    @either.flush
+    @twice.flush
+    @ledger.charge!(1)
+    @tapped.flush
+    @outside.info("x")
+    @b.flush
+    @unset.flush
+  end
+
+  def self.klass
+    @writer.flush
+  end
+end
+
+module Mixin
+  def mixed
+    @writer.flush
+  end
+end
+`,
+};
+
+let root = '';
+let cg: CodeGraph | undefined;
+let savedKernel: string | undefined;
+
+afterEach(() => {
+  cg?.close();
+  cg = undefined;
+  if (root) fs.rmSync(root, { recursive: true, force: true });
+  root = '';
+  if (savedKernel === undefined) delete process.env.CODEGRAPH_KERNEL;
+  else process.env.CODEGRAPH_KERNEL = savedKernel;
+});
+
+async function index(backend: string): Promise<CodeGraph> {
+  savedKernel = process.env.CODEGRAPH_KERNEL;
+  if (backend === 'wasm') process.env.CODEGRAPH_KERNEL = '0';
+  else delete process.env.CODEGRAPH_KERNEL;
+  root = fs.mkdtempSync(path.join(os.tmpdir(), 'cg-ruby-ivar-'));
+  for (const [rel, content] of Object.entries(FILES)) {
+    fs.mkdirSync(path.dirname(path.join(root, rel)), { recursive: true });
+    fs.writeFileSync(path.join(root, rel), content);
+  }
+  cg = await CodeGraph.init(root, { index: true });
+  return cg;
+}
+
+/** `<qualified name>:<line>` of every `calls` edge written on `line` of `file`. */
+function callsAt(graph: CodeGraph, file: string, line: number): string[] {
+  const ids = graph.getNodesInFile(file).map((n) => n.id);
+  return graph
+    .getOutgoingEdgesFrom(ids)
+    .filter((e) => e.kind === 'calls' && e.line === line)
+    .map((e) => {
+      const t = graph.getNode(e.target)!;
+      return `${t.qualifiedName}:${t.startLine}`;
+    })
+    .sort();
+}
+
+const backends = [
+  ['kernel', getKernel() === null],
+  ['wasm', false],
+] as const;
+
+describe.each(backends)('Ruby calls on an instance variable initialize types (%s)', (backend, skip) => {
+  it.skipIf(skip)('links the method of the class initialize constructs', async () => {
+    const graph = await index(backend);
+    const holder = 'app/holder.rb';
+    expect(callsAt(graph, holder, 19)).toEqual(['CacheWriter::flush:2']);
+    expect(callsAt(graph, holder, 20)).toEqual(['CacheWriter::ready?:6']);
+    // `||=` with its arguments over several lines.
+    expect(callsAt(graph, holder, 21)).toEqual(['OtherWriter::flush:12']);
+    // A namespaced class, built with a block: the method its superclass has.
+    expect(callsAt(graph, holder, 24)).toEqual(['Billing::Card::charge!:19']);
+  });
+
+  it.skipIf(skip)('types nothing initialize leaves ambiguous', async () => {
+    const graph = await index(backend);
+    const holder = 'app/holder.rb';
+    for (const line of [22, 23, 25, 27, 28]) expect(callsAt(graph, holder, line)).toEqual([]);
+  });
+
+  it.skipIf(skip)('links nothing for a class the project does not declare', async () => {
+    const graph = await index(backend);
+    expect(callsAt(graph, 'app/holder.rb', 26)).toEqual([]);
+  });
+
+  it.skipIf(skip)("leaves a class method's and a module's instance variables untyped", async () => {
+    const graph = await index(backend);
+    expect(callsAt(graph, 'app/holder.rb', 32)).toEqual([]);
+    expect(callsAt(graph, 'app/holder.rb', 38)).toEqual([]);
+  });
+});

src/resolution/ruby-constant-paths.ts

@@ -20,11 +20,16 @@
  * A call on what `.new` made (`ChargeCard.new(card).charge!`, recorded as
  * `new ChargeCard().charge!`) resolves here too, bare constant or path: an
  * instance method of the class the lookup finds, or nothing (#684).
+ *
+ * So does a call on an instance variable an instance method makes
+ * (`@writer.flush`), when the class's `initialize` gives the variable an
+ * instance of one class, `@writer = CacheWriter.new(…)`, and nothing else.
  */
 import type { Node } from '../types';
 import type { ResolutionContext, ResolvedRef, UnresolvedRef } from './types';
 import { isTestPath } from '../search/query-utils';
 import { rubyAncestry, rubyMethodsOfKindFirst, type RubyMethodKind } from './name-matcher';
+import { stripCommentsForRegex } from './strip-comments';
 
 /** `A::B`, `::A`, `::A::B`: a constant path. Group 1 is a leading `::`. */
 const CONSTANT_PATH = /^(::)?([A-Z]\w*(?:::[A-Z]\w*)*)$/;
@@ -35,6 +40,8 @@ const CONSTANT_PATH_CALL = /^(::)?([A-Z]\w*(?:::[A-Z]\w*)*)\.([A-Za-z_]\w*[?!=]?
  * made, as the extractor records it (#684). Group 3 is the method.
  */
 const CONSTRUCTED_CALL = /^new (::)?([A-Z]\w*(?:::[A-Z]\w*)*)\(\)\.([A-Za-z_]\w*[?!=]?)$/;
+/** `@writer.flush`: a call on an instance variable. Group 1 is the variable's name, group 2 the method. */
+const IVAR_CALL = /^@(\w+)\.([A-Za-z_]\w*[?!=]?)$/;
 
 /** What a Ruby constant is in the index: a class, a module, or a value (`RATES = {…}` is a `variable`). */
 const NAMESPACE_KINDS: ReadonlySet<string> = new Set(['class', 'module']);
@@ -68,6 +75,17 @@ export function matchRubyConstantPathRef(ref: UnresolvedRef, context: Resolution
       const method = cls?.kind === 'class' ? methodOf(cls.qualifiedName, made[3]!, ref, context, 'instance') : null;
       return method ? { original: ref, targetNodeId: method.node.id, confidence: method.own ? 0.9 : 0.85, resolvedBy: 'instance-method' } : null;
     }
+    // A call on an instance variable, typed by the class's `initialize`: an
+    // instance method of the class it is given there, or nothing when that
+    // class is not the project's. A variable `initialize` doesn't type is
+    // left to the other strategies, as before.
+    const ivar = IVAR_CALL.exec(ref.referenceName);
+    if (ivar) {
+      const cls = ivarClass(ivar[1]!, ref, context);
+      if (cls === undefined) return undefined;
+      const method = cls ? methodOf(cls.qualifiedName, ivar[2]!, ref, context, 'instance') : null;
+      return method ? { original: ref, targetNodeId: method.node.id, confidence: method.own ? 0.85 : 0.8, resolvedBy: 'instance-method' } : null;
+    }
     const m = CONSTANT_PATH_CALL.exec(ref.referenceName);
     if (!m || !isNamespaced(m)) return undefined;
     const owner = constantsNamed(m[2]!, m[1] !== undefined, ref, context)[0];
@@ -106,13 +124,129 @@ function constantsNamed(path: string, absolute: boolean, ref: UnresolvedRef, con
 
 /** The qualified name of the innermost class or module around `ref`, or '' at the top level of its file. */
 function lexicalScope(ref: UnresolvedRef, context: ResolutionContext): string {
+  return innermostNamespace(ref, context)?.qualifiedName ?? '';
+}
+
+/** The innermost class or module around `ref`, if any. */
+function innermostNamespace(ref: UnresolvedRef, context: ResolutionContext): Node | undefined {
   let inner: Node | undefined;
   for (const n of context.getNodesInFile(ref.filePath)) {
     if (!NAMESPACE_KINDS.has(n.kind) || n.language !== 'ruby' || n.startLine > ref.line || n.endLine < ref.line) continue;
     // Of the ranges around a point, the innermost starts last.
     if (!inner || n.startLine > inner.startLine || (n.startLine === inner.startLine && n.startColumn > inner.startColumn)) inner = n;
   }
-  return inner?.qualifiedName ?? '';
+  return inner;
+}
+
+const IVAR_CLASSES = new WeakMap<ResolutionContext, Map<string, Node | null | undefined>>();
+
+/**
+ * The class of what the instance variable `@name` holds where `ref` is
+ * written, an instance method of a class: read from that class's
+ * `initialize`, where every assignment to the variable must be
+ * `@name = Const.new(…)` (or `||=`) of one constant. Then it is the class
+ * Ruby's lookup finds for that constant from there, or null when the
+ * project declares no such class. Undefined, for the other strategies,
+ * when the call is not in an instance method of a class, or `initialize`
+ * doesn't assign the variable, or assigns it anything else.
+ */
+function ivarClass(name: string, ref: UnresolvedRef, context: ResolutionContext): Node | null | undefined {
+  const caller = context.getNodeById?.(ref.fromNodeId);
+  if (!caller || caller.kind !== 'method' || caller.language !== 'ruby' || caller.isStatic === true) return undefined;
+  const owner = innermostNamespace(ref, context);
+  if (!owner || owner.kind !== 'class') return undefined;
+  let memo = IVAR_CLASSES.get(context);
+  if (!memo) IVAR_CLASSES.set(context, (memo = new Map()));
+  const key = `${owner.qualifiedName}\0${name}`;
+  if (memo.has(key)) return memo.get(key);
+  const found = ivarClassFromInitialize(name, owner.qualifiedName, context);
+  memo.set(key, found);
+  return found;
+}
+
+function ivarClassFromInitialize(name: string, owner: string, context: ResolutionContext): Node | null | undefined {
+  let written: { path: string; absolute: boolean; file: string; line: number } | undefined;
+  for (const init of context.getNodesByQualifiedName(`${owner}::initialize`)) {
+    if (init.kind !== 'method' || init.language !== 'ruby' || init.isStatic === true) continue;
+    const lines = context.getFileLines?.(init.filePath) ?? context.readFile(init.filePath)?.split(/\r?\n/);
+    if (!lines) return undefined;
+    const body = stripCommentsForRegex(lines.slice(init.startLine - 1, init.endLine).join('\n'), 'ruby');
+    const assigned = constructedAssignments(body, name);
+    if (!assigned) return undefined;
+    for (const a of assigned) {
+      if (written && (written.path !== a.path || written.absolute !== a.absolute)) return undefined;
+      written ??= { path: a.path, absolute: a.absolute, file: init.filePath, line: init.startLine + a.line };
+    }
+  }
+  if (!written) return undefined;
+  const at: UnresolvedRef = {
+    fromNodeId: '', referenceName: written.path, referenceKind: 'references',
+    line: written.line, column: 0, filePath: written.file, language: 'ruby',
+  };
+  const cls = constantsNamed(written.path, written.absolute, at, context)[0];
+  return cls?.kind === 'class' ? cls : null;
+}
+
+/**
+ * Each assignment to `@name` in `body` (an `initialize`, comments blanked):
+ * the constant whose `.new` it assigns, and the assignment's line within
+ * `body`. Undefined when one of them is anything else: not at the start of
+ * a statement (`@a, @b = …`), or not a lone `Const.new(…)` (`Foo.new.tap
+ * {…}`, `writer || Foo.new`, `build_writer`).
+ */
+function constructedAssignments(body: string, name: string): { path: string; absolute: boolean; line: number }[] | undefined {
+  const out: { path: string; absolute: boolean; line: number }[] = [];
+  const target = new RegExp(String.raw`(?<![@\w])@${name}\s*(?:\|\||&&|\*\*|<<|>>|[-+*/%|&^])?=(?![=~>])`, 'g');
+  for (const m of body.matchAll(target)) {
+    const at = m.index!;
+    const statementStart = Math.max(body.lastIndexOf('\n', at - 1), body.lastIndexOf(';', at - 1)) + 1;
+    if (body.slice(statementStart, at).trim() !== '') return undefined;
+    const value = constructedValue(body, at + m[0].length);
+    if (!value) return undefined;
+    let line = 0;
+    for (let i = body.indexOf('\n'); i !== -1 && i < at; i = body.indexOf('\n', i + 1)) line++;
+    out.push({ ...value, line });
+  }
+  return out;
+}
+
+/**
+ * The constant a value written at `from` constructs when it is a lone
+ * `Const.new`, `A::B.new(…)` or `::Foo.new …`: its arguments in parentheses
+ * or without them, a block, or a statement modifier may follow, but nothing
+ * the value is passed through (`.tap`, `||`). Null otherwise.
+ */
+function constructedValue(s: string, from: number): { path: string; absolute: boolean } | null {
+  const m = /^\s*(::)?([A-Z]\w*(?:::[A-Z]\w*)*)\s*\.\s*new(?![\w?!])/.exec(s.slice(from));
+  if (!m) return null;
+  let i = from + m[0].length;
+  if (s[i] === '(') {
+    i = skipParens(s, i);
+    if (i < 0) return null;
+  }
+  // The end of the statement, a block, or (after a space) a modifier or
+  // arguments without parentheses: anything but a chain or an operator.
+  const next = s[i];
+  const ok = next === undefined || next === '\n' || next === ';' || next === '{' ||
+    /^[ \t]+(?:$|\n|;|\{|[\w:@$"'[])/.test(s.slice(i, i + 200));
+  // A chain may go on at the start of the next line: `Foo.new\n  .tap {…}`.
+  const continued = /^[ \t]*\n\s*&?\./.test(s.slice(i, i + 200));
+  return ok && !continued ? { path: m[2]!, absolute: m[1] !== undefined } : null;
+}
+
+/** The index past the parenthesized group opening at `i`, Ruby strings inside it skipped; -1 when it never closes. */
+function skipParens(s: string, i: number): number {
+  let depth = 0;
+  for (let j = i; j < s.length; j++) {
+    const c = s[j]!;
+    if (c === '"' || c === "'") {
+      for (j++; j < s.length && s[j] !== c; j++) if (s[j] === '\\') j++;
+      continue;
+    }
+    if (c === '(') depth++;
+    else if (c === ')' && --depth === 0) return j + 1;
+  }
+  return -1;
 }
 
 /**