Skip to content

Use quick_stat with full_major instead of stat (which will be deprecated) - #60

Open
dinosaure wants to merge 1 commit into
mainfrom
gc-stats
Open

Use quick_stat with full_major instead of stat (which will be deprecated)#60
dinosaure wants to merge 1 commit into
mainfrom
gc-stats

Conversation

@dinosaure

Copy link
Copy Markdown
Member

See this PR on ocaml/ocaml for more explanation:
ocaml/ocaml#12023

/cc @damiendoligez (to confirm if it's the right usage) and /cc @Julow if it does not disturb your metrics.

@Julow

Julow commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

I think this doesn't give the right result.
Here's a repro:

I get 1844w/run (varying in each runs) with this PR, 3000w/run (always) with minor_allocated_precise (#54) and 1658w/run (varying too) with the released version. I think 3000 is the right number ?

open Bechamel
open Toolkit

let minor_allocated_precise =
  let module Ext = struct
    type witness = unit

    let load () = ()
    let unload () = ()
    let make () = ()
    let get () = Gc.minor_words ()
    let label () = "minor-allocated"
    let unit () = "word"
  end in
  let ext = Measure.register (module Ext) in
  Measure.instance (module Ext) ext

let input = List.init 1000 (fun i -> i)

let bench =
  Test.make ~name:"map"
    (Staged.stage @@ fun () -> ignore (List.map (fun x -> x + 1) input))

(* let instances = [ Instance.minor_allocated; minor_allocated_precise ] *)
let instances =
  if false then [ Instance.minor_allocated ] else [ minor_allocated_precise ]

let () =
  let cfg = Benchmark.cfg ~limit:2000 ~quota:(Time.second 0.1) () in
  let raw = Benchmark.all cfg instances bench in
  let ols =
    Analyze.ols ~bootstrap:0 ~r_square:true ~predictors:[| Measure.run |]
  in
  List.iter
    (fun inst ->
      Hashtbl.iter
        (fun name ols ->
          Format.printf "%s %s %a@." (Measure.label inst) name Analyze.OLS.pp
            ols)
        (Analyze.all ols inst raw))
    instances

@dinosaure

Copy link
Copy Markdown
Member Author

Hmmhmm, so to be clear, this PR and what you propose on #54 is about two different questions:

  • for this PR, the question is more how we can stabilize the major heap before to run a benchmark
  • what you proposed on Minor_allocated: Use Gc.minor_words #54 is indeed interesting and the solution I would take to get metrics

So the question is much more: if this PR and #54 together produce a good result or not (but it's not like choose one or the other). From your code, I can reproduce this with the merge of #54 and this PR:

$ ./a.out
minor-allocated map { minor-allocated per run = 3000.000866; r² = Some 1 }

With bechamel.0.5.0, I can also have the same result, not sure how to reproduce the bad case 😕. I get also the same result with the upstream version.

$ ./a.out
minor-allocated map { minor-allocated per run = 3000.000946; r² = Some 1 }

With only #54, I have this result:

$ ./a.out
minor-allocated map { minor-allocated per run = 3000.001034; r² = Some 1 }

@dinosaure

Copy link
Copy Markdown
Member Author

And, btw, it will be nice to also look into #55 🥲

@Julow

Julow commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

I confirm the bad behavior with bechamel 0.5.0 and ocaml 5.3. I was expecting that the current PR fixes the same bug as #54 because both use Gc.quick_stat. Maybe that's explained by my ocaml version ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants