Skip to content

Inflated geometric factors storage for affine elements #314

Closed

Description

@inducer

In #310 (comment), @kaushikcfd says:

I noticed a subtle and sub-optimal behavior that is worth noting. If we are using PyOpenCLActx as a setup actx and PytatoActx as an operational actx, then the frozen geometric factors (such as normals) would be stored at per-DOF even for affine geometry mappings. Such use-cases will bump memory footprint.

If true, this wouldn't just increase memory footprint, but also memory bandwidth requirements.

cc @majosm @matthiasdiener

Activity

  1. majosm commented on Aug 10, 2023

    @majosm
    Collaborator

    I'm told that we don't use a separate array context for init anymore in Y3, so it looks like this isn't affecting us at the moment.

  2. inducer commented on Aug 10, 2023

    @inducer
    OwnerAuthor

    @kaushikcfd Could you elaborate on the circumstance under which you've observed this? Do you happen to know why this happens?

  3. kaushikcfd commented on Aug 11, 2023

    @kaushikcfd
    Collaborator

    I did not observe this in any production-related drivers. But I was writing some tests for #310 and noticed this behavior and thought sharing would be essential so that others do not repeat the same mistake.

    I have attached a minimal reproducer script for posterity.

    import grudge
    import pyopencl as cl
    from meshmode.discretization.connection import FACE_RESTR_INTERIOR
    
    from grudge.array_context import get_reasonable_array_context_class
    from meshmode.mesh.generation import generate_regular_rect_mesh
    
    ctx = cl.create_some_context()
    cq = cl.CommandQueue(ctx)
    
    ref_actx = get_reasonable_array_context_class(lazy=False, distributed=False)(cq)
    actx = get_reasonable_array_context_class(lazy=True, distributed=False)(cq)
    
    mesh = generate_regular_rect_mesh(a=(-0.5, -0.5),
                                      b=(0.5, 0.5),
                                      nelements_per_axis=(16, 16))
    
    dcoll = grudge.make_discretization_collection(ref_actx, mesh, order=4)
    nx, ny = actx.thaw(dcoll.normal(FACE_RESTR_INTERIOR))
    
    assert nx[0].shape[-1] == 1  # fails
  4. inducer commented on Aug 11, 2023

    @inducer
    OwnerAuthor

    Thanks for providing the reproducer. AFAICT, this affects only the normals. The main issue is that here:

    return self._setup_actx.freeze(normal(self._setup_actx, self, dd))

    the setup actx is unconditionally used to populate the normal cache, making this an interface bug more than a "bug bug". Perhaps the most reasonable response is to deprecate DiscretizationCollection.normal. grudge.geometry.normal takes an actx and does not have this issue (and it's more convenient to use because there's no freeze/thaw cycle needed).

  5. added a commit that references this issue on Aug 11, 2023
    2bb1230
  6. inducer commented on Aug 11, 2023

    @inducer
    OwnerAuthor
  7. added a commit that references this issue on Mar 6, 2024
    294a4cc
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions