5.7. Random Pile’o’Tools

Provide some utilities for building simulation applications.

5.7.1. General utilities

mirgecom.simutil.check_step(step, interval)[source]

Check step number against a user-specified interval.

Utility is used typically for visualization.

  • Negative numbers mean ‘never visualize’.

  • Zero means ‘always visualize’.

Useful for checking whether the current step is an output step, or anything else that occurs on fixed intervals.

mirgecom.simutil.get_sim_timestep(dcoll, state, t, dt, cfl, t_final=0.0, constant_cfl=False, local_dt=False, fluid_dd=DOFDesc(domain_tag=VolumeDomainTag(tag=<class 'grudge.dof_desc.VTAG_ALL'>), discretization_tag=<class 'grudge.dof_desc.DISCR_TAG_BASE'>))[source]

Return the maximum stable timestep for a typical fluid simulation.

This routine returns a constraint-limited timestep size for a fluid simulation. The returned timestep will be constrained by the specified Courant-Friedrichs-Lewy number, cfl, and the simulation max simulated time limit, t_final, and subject to the user’s optional settings.

The local fluid timestep, \(\delta{t}_l\), is computed by get_viscous_timestep(). Users are referred to that routine for the details of the local timestep.

With the remaining simulation time \(\Delta{t}_r = \left(\mathit{t\_final}-\mathit{t}\right)\), three modes are supported for the returned timestep, \(\delta{t}\):

  • “Constant DT” mode (default): \(\delta{t} = \mathbf{\text{min}} \left(\textit{dt},~\Delta{t}_r\right)\)

  • “Constant CFL” mode (constant_cfl=True): \(\delta{t} = \mathbf{\text{min}}\left(\mathbf{\text{global\_min}}\left(\delta{t}\_l\right) ,~\Delta{t}_r\right)\)

  • “Local DT” mode (local_dt=True): \(\delta{t} = \mathbf{\text{cell\_local\_min}} \left(\delta{t}_l\right)\)

Note that for “Local DT” mode, t_final is ignored, and a DOFArray containing the local cfl-limited timestep, where \(\mathbf{\text{cell\_local\_min}}\left(\delta{t}\_l\right)\) is defined as the minimum over the cell collocation points. This mode is useful for stepping to convergence of steady-state solutions.

Important

For “Constant CFL” mode, this routine calls the collective nodal_min() on the inside which involves MPI collective functions. Thus all MPI ranks on the DiscretizationCollection must call this routine collectively when using “Constant CFL” mode.

Parameters:

  • dcoll (DiscretizationCollection) – The DG discretization collection to use

  • state (FluidState) – The full fluid conserved and thermal state

  • t (float) – Current time

  • t_final (float) – Final time

  • dt (float) – The current timestep

  • cfl (float) – The current CFL number

  • constant_cfl (bool) – True if running constant CFL mode

  • local_dt (bool) – True if running local DT mode. False by default.

  • fluid_dd (grudge.dof_desc.DOFDesc) – the DOF descriptor of the discretization on which state lives. Must be a volume on the base discretization.

Returns:

The global maximum stable DT based on a viscous fluid.

Return type:

float or DOFArray

mirgecom.simutil.write_visfile(dcoll, io_fields, visualizer, vizname, step=0, t=0, overwrite=False, vis_timer=None, comm=None)[source]

Write parallel VTK output for the fields specified in io_fields.

This routine writes a parallel-compatible unstructured VTK visualization file set in (vtu/pvtu) format. One file per MPI rank is written with the following naming convention: vizname*_*step*_<mpi-rank>.vtu, and a single file manifest with naming convention: *vizname*_*step.pvtu. Users are advised to visualize the data using _Paraview_, _VisIt_, or other VTU-compatible visualization software by opening the PVTU files.

Note

This is a collective routine and must be called by all MPI ranks.

Parameters:

  • visualizer – A meshmode.discretization.visualization.Visualizer VTK output object.

  • io_fields – List of tuples indicating the (name, data) for each field to write.

  • vizname (str) – Root part of the visualization file name to write

  • step (int) – The step number to use in the file names

  • t (float) – The simulation time to write into the visualization files

  • overwrite (bool) – Option whether to overwrite existing files (True) or fail if files exist (False=default).

  • comm – An MPI Communicator is required for parallel writes. If no mpi_communicator is provided, then the write is assumed to be serial. (deprecated behavior: pull an MPI communicator from the discretization collection. This will stop working in Fall 2022.)

mirgecom.simutil.global_reduce(local_values, op, *, comm=None)[source]

Perform a global reduction (allreduce if MPI comm is provided).

This routine is a convenience wrapper for the MPI AllReduce operation that also works outside of an MPI context.

Note

This is a collective routine and must be called by all MPI ranks.

Parameters:

  • local_values – The (mpi4py-compatible) value or array of values on which the reduction operation is to be performed.

  • op (str) – Reduction operation to be performed. Must be one of “min”, “max”, “sum”, “prod”, “lor”, or “land”.

  • comm – Optional parameter specifying the MPI communicator on which the reduction operation (if any) is to be performed

Returns:

Returns the result of the reduction operation on local_values

Return type:

Any ( like local_values )

mirgecom.simutil.get_reasonable_memory_pool(ctx, queue, force_buffer=False, force_non_pool=False)[source]

Return an SVM or buffer memory pool based on what the device supports.

By default, it prefers SVM allocations over CL buffers, and memory pools over direct allocations.

Parameters:
mirgecom.simutil.queue_finish(queue)[source]

Finish the CL command queue if it is not None.

Parameters:

queue (CommandQueue | None)

Return type:

None

5.7.2. Diagnostic utilities

mirgecom.simutil.compare_fluid_solutions(dcoll, red_state, blue_state, *, dd=DOFDesc(domain_tag=VolumeDomainTag(tag=<class 'grudge.dof_desc.VTAG_ALL'>), discretization_tag=<class 'grudge.dof_desc.DISCR_TAG_BASE'>))[source]

Return inf norm of (red_state - blue_state) for each component.

Note

This is a collective routine and must be called by all MPI ranks.

mirgecom.simutil.componentwise_norms(dcoll, fields, order=inf, *, dd=DOFDesc(domain_tag=VolumeDomainTag(tag=<class 'grudge.dof_desc.VTAG_ALL'>), discretization_tag=<class 'grudge.dof_desc.DISCR_TAG_BASE'>))[source]

Return the order-norm for each component of fields.

Note

This is a collective routine and must be called by all MPI ranks.

mirgecom.simutil.max_component_norm(dcoll, fields, order=inf, *, dd=DOFDesc(domain_tag=VolumeDomainTag(tag=<class 'grudge.dof_desc.VTAG_ALL'>), discretization_tag=<class 'grudge.dof_desc.DISCR_TAG_BASE'>))[source]

Return the max order-norm over the components of fields.

Note

This is a collective routine and must be called by all MPI ranks.

mirgecom.simutil.check_naninf_local(dcoll, dd, field)[source]

Return True if there are any NaNs or Infs in the field.

Parameters:
Return type:

bool

mirgecom.simutil.check_range_local(dcoll, dd, field, min_value, max_value)[source]

Return the values that are outside the range [min_value, max_value].

Parameters:
Return type:

List[float]

mirgecom.simutil.boundary_report(dcoll, boundaries, outfile_name, *, dd=DOFDesc(domain_tag=VolumeDomainTag(tag=<class 'grudge.dof_desc.VTAG_ALL'>), discretization_tag=<class 'grudge.dof_desc.DISCR_TAG_BASE'>), mesh=None)[source]

Generate a report of the mesh boundaries.

5.7.3. Mesh and element utilities

mirgecom.simutil.geometric_mesh_partitioner(mesh, num_ranks=None, *, nranks_per_axis=None, auto_balance=False, imbalance_tolerance=0.01, debug=False)[source]

Partition a mesh uniformly along the X coordinate axis.

The intent is to partition the mesh uniformly along user-specified directions. In this current interation, the capability is demonstrated by splitting along the X axis.

Parameters:

  • mesh (meshmode.mesh.Mesh) – The serial mesh to partition

  • num_ranks (int) – The number of partitions to make (deprecated)

  • nranks_per_axis (numpy.ndarray) – How many partitions per specified axis.

  • auto_balance (bool) – Indicates whether to perform automatic balancing. If true, the partitioner will try to balance the number of elements over the partitions.

  • imbalance_tolerance (float) – If auto_balance is True, this parameter indicates the acceptable relative difference to the average number of elements per partition. It defaults to balance within 1%.

  • debug (bool) – En/disable debugging/diagnostic print reporting.

Returns:

elem_to_rank – Array indicating the MPI rank for each element

Return type:

numpy.ndarray

mirgecom.simutil.distribute_mesh(comm, get_mesh_data, partition_generator_func=None, logmgr=None, num_per_batch=None)[source]

Distribute a mesh among all ranks in comm.

Retrieve the global mesh data with the user-supplied function get_mesh_data, partition the mesh, and distribute it to every rank in the provided MPI communicator comm.

Note

This is a collective routine and must be called by all MPI ranks.

Parameters:

  • comm – MPI communicator over which to partition the mesh

  • get_mesh_data – Callable of zero arguments returning mesh or (mesh, tag_to_elements, volume_to_tags), where mesh is a meshmode.mesh.Mesh, tag_to_elements is a dict mapping mesh volume tags to numpy.ndarrays of element numbers, and volume_to_tags is a dict that maps volumes in the resulting distributed mesh to volume tags in tag_to_elements.

  • partition_generator_func – Optional callable that takes mesh, tag_to_elements, and comm’s size, and returns a numpy.ndarray indicating to which rank each element belongs.

Returns:

  • local_mesh_data (meshmode.mesh.Mesh or dict) – If the result of calling get_mesh_data specifies a single volume, local_mesh_data is the local mesh. If it specifies multiple volumes, local_mesh_data will be a dict mapping volume tags to tuples of the form (local_mesh, local_tag_to_elements).

  • global_nelements (int) – The number of elements in the global mesh

mirgecom.simutil.get_number_of_tetrahedron_nodes(dim, order, include_faces=False)[source]

Get number of nodes (modes) in dim Tetrahedron of order.

mirgecom.simutil.get_box_mesh(dim, a, b, n, t=None, periodic=None, tensor_product_elements=False, **kwargs)[source]

Create a rectangular “box” like mesh with tagged boundary faces.

The resulting mesh has boundary tags “-i” and “+i” for i=1,…,dim corresponding to lower and upper faces normal to coordinate dimension i.

Parameters:

  • dim (int) – The mesh topological dimension

  • a (float or tuple) – The coordinates of the lower corner of the box. If scalar-valued, gets promoted to a uniform tuple.

  • b (float or tuple) – The coordinates of the upper corner of the box. If scalar-valued, gets promoted to a uniform tuple.

  • n (int or tuple) – The number of elements along a given dimension. If scalar-valued, gets promoted to a uniform tuple.

  • t (str or None) – The mesh type. See meshmode.mesh.generation.generate_box_mesh() for details.

  • periodic (bool or tuple or None) – Indicates whether the mesh is periodic in a given dimension. If scalar-valued, gets promoted to a uniform tuple.

Returns:

The generated box mesh.

Return type:

meshmode.mesh.Mesh

mirgecom.simutil.interdecomposition_mapping(target_decomp, source_decomp)[source]

Return a mapping of which partitions to source for the target decomp.

mirgecom.simutil.interdecomposition_overlap(target_decomp_map, source_decomp_map, return_parts=None)[source]

Map element indices for overlapping, disparate decompositions.

For each (or optionally selected) target parts, this routine returns a dictionary keyed by overlapping remote partitions from the source decomposition, the value of which is a map from the target-part-specific local indexes to the source-part-specific local index of for the corresponding element.

Example dictionary structure:

{
  targ_part_1 : {
    src_part_1 : { local_el_index : remote_el_index, ... },
    src_part_2 : { local_el_index : remote_el_index, ... },
    ...
  },
  targ_part_2 : { ... },
  ...
}

This data structure is useful for mapping the solution data from the old decomp pkl restart files to the new decomp solution arrays.

mirgecom.simutil.invert_decomp(decomp_map)[source]

Return a list of global elements for each partition.

mirgecom.simutil.multivolume_interdecomposition_overlap(src_decomp_map, trg_decomp_map, src_multivol_decomp_map, trg_multivol_decomp_map, return_ranks=None)[source]

Construct local-to-local index mapping for overlapping decompositions.

Parameters:

  • src_decomp_map (dict) – Source decomposition map {rank: [elements]}.

  • trg_decomp_map (dict) – Target decomposition map {rank: [elements]}.

  • src_multivol_decomp_map (dict) – Source multivolume decomposition map {PartID: np.array(elements)}.

  • trg_multivol_decomp_map (dict) – Target multivolume decomposition map {PartID: np.array(elements)}.

Returns:

A dictionary with the following structure

{
    trg_partid: {
        src_partid: {
            trg_local_el_index: src_local_el_index
        }
    }
}

Return type:

dict

mirgecom.simutil.copy_mapped_dof_array_data(trg_dof_array, src_dof_array, index_map)[source]

Copy data between DOFArrays from disparate meshes.

5.7.4. Simulation support utilities

mirgecom.simutil.configurate(config_key, config_object=None, default_value=None)[source]

Return a configured item from a configuration object.

5.7.5. File comparison utilities

mirgecom.simutil.compare_files_vtu(first_file, second_file, file_type, tolerance=1e-12, field_tolerance=None)[source]

Compare files of vtu type.

Parameters:

  • first_file (str) – First file to compare

  • second_file (str) – Second file to compare

  • file_type (str) – Vtu files

  • tolerance (float) – Max acceptable absolute difference

  • field_tolerance (Dict[str, float] | None) – Dictionary of individual field tolerances

Returns:

  • True – If it passes the files contain data within the given tolerance.

  • False – If it fails the files contain data outside the given tolerance.

Return type:

None

mirgecom.simutil.compare_files_xdmf(first_file, second_file, tolerance=1e-12)[source]

Compare files of xdmf type.

Parameters:

  • first_file (str) – First file to compare

  • second_file (str) – Second file to compare

  • file_type – Xdmf files

  • tolerance (float) – Max acceptable absolute difference

Returns:

  • True – If it passes the file type test or contains same data.

  • False – If it fails the file type test or contains different data.

mirgecom.simutil.compare_files_hdf5(first_file, second_file, tolerance=1e-12)[source]

Compare files of hdf5 type.

Parameters:

  • first_file (str) – First file to compare

  • second_file (str) – Second file to compare

  • file_type – Hdf5 files

  • tolerance (float) – Max acceptable absolute difference

Returns:

  • True – If it passes the file type test or contains same data.

  • False – If it fails the file type test or contains different data.

5.7.6. Exceptions

class mirgecom.simutil.SimulationConfigurationError[source]

Simulation physics configuration or parameters error.

class mirgecom.simutil.ApplicationOptionsError[source]

Application command-line options error.

class mirgecom.simutil.SimulationRuntimeError[source]

General simulation runtime error.

Useful bits and bobs.

class mirgecom.utils.StatisticsAccumulator(scale_factor=1)[source]

Class that provides statistical functions for multiple values.

Parameters:

scale_factor (float)

__init__(scale_factor=1)[source]

Initialize an empty StatisticsAccumulator object.

Parameters:

scale_factor (float) – Scale returned statistics by this factor.

Return type:

None

add_value(v)[source]

Add a new value to the statistics.

Parameters:

v (float)

Return type:

None

sum()[source]

Return the sum of added values.

Return type:

float | None

mean()[source]

Return the mean of added values.

Return type:

float | None

max()[source]

Return the max of added values.

Return type:

float | None

min()[source]

Return the min of added values.

Return type:

float | None

num_values: int
mirgecom.utils.asdict_shallow(dc_instance)[source]

Convert a dataclass into a dict.

What dataclasses.asdict() should have been: no recursion, no deep copy. Simply turn one layer of a dataclass into a dict.

Return type:

dict

mirgecom.utils.force_evaluation(actx, x)[source]

Force evaluation of a (possibly lazy) array.

mirgecom.utils.force_compile(actx, f, *args)[source]

Force compilation of f with args.

mirgecom.utils.normalize_boundaries(boundaries)[source]

Normalize the keys of boundaries.

Promotes boundary tags to grudge.dof_desc.BoundaryDomainTag.

mirgecom.utils.project_from_base(dcoll, tgt_dd, field)[source]

Project field from DISCR_TAG_BASE to the same discr. as tgt_dd.

mirgecom.utils.mask_from_elements(dcoll, dd, actx, elements)[source]

Get a DOFArray mask corresponding to elements.

Returns:

mask – A DOF array containing \(1\) for elements that are in elements and \(0\) for elements that aren’t.

Return type:

meshmode.dof_array.DOFArray

Provide some utilities for handling ArrayContexts.

mirgecom.array_context.get_reasonable_array_context_class(*, lazy, distributed, profiling, numpy=False)[source]

Return a ArrayContext with the given constraints.

Parameters:
Return type:

Type[ArrayContext]

mirgecom.array_context.actx_class_is_lazy(actx_class)[source]

Return True if actx_class is lazy.

Parameters:

actx_class (Type[ArrayContext])

Return type:

bool

mirgecom.array_context.actx_class_is_eager(actx_class)[source]

Return True if actx_class is eager.

Parameters:

actx_class (Type[ArrayContext])

Return type:

bool

mirgecom.array_context.actx_class_is_profiling(actx_class)[source]

Return True if actx_class has profiling enabled.

Parameters:

actx_class (Type[ArrayContext])

Return type:

bool

mirgecom.array_context.actx_class_is_numpy(actx_class)[source]

Return True if actx_class is numpy-based.

Parameters:

actx_class (Type[ArrayContext])

Return type:

bool

mirgecom.array_context.actx_class_is_distributed(actx_class)[source]

Return True if actx_class is distributed.

Parameters:

actx_class (Type[ArrayContext])

Return type:

bool

mirgecom.array_context.initialize_actx(actx_class, comm=None, *, use_axis_tag_inference_fallback=False, use_einsum_inference_fallback=False)[source]

Initialize a new ArrayContext based on actx_class.

Parameters:
Return type:

ArrayContext