hwlocality_membinding(3) Hardware Locality (hwloc) hwlocality_membinding(3)NAME
hwlocality_membinding - Memory binding
Enumerations
enum hwloc_membind_policy_t { HWLOC_MEMBIND_DEFAULT,
HWLOC_MEMBIND_FIRSTTOUCH, HWLOC_MEMBIND_BIND,
HWLOC_MEMBIND_INTERLEAVE, HWLOC_MEMBIND_REPLICATE,
HWLOC_MEMBIND_NEXTTOUCH, HWLOC_MEMBIND_MIXED }
enum hwloc_membind_flags_t { HWLOC_MEMBIND_PROCESS,
HWLOC_MEMBIND_THREAD, HWLOC_MEMBIND_STRICT, HWLOC_MEMBIND_MIGRATE,
HWLOC_MEMBIND_NOCPUBIND }
Functions
int hwloc_set_membind_nodeset (hwloc_topology_t topology,
hwloc_const_nodeset_t nodeset, hwloc_membind_policy_t policy, int
flags)
int hwloc_set_membind (hwloc_topology_t topology, hwloc_const_cpuset_t
cpuset, hwloc_membind_policy_t policy, int flags)
int hwloc_get_membind_nodeset (hwloc_topology_t topology,
hwloc_nodeset_t nodeset, hwloc_membind_policy_t *policy, int flags)
int hwloc_get_membind (hwloc_topology_t topology, hwloc_cpuset_t
cpuset, hwloc_membind_policy_t *policy, int flags)
int hwloc_set_proc_membind_nodeset (hwloc_topology_t topology,
hwloc_pid_t pid, hwloc_const_nodeset_t nodeset,
hwloc_membind_policy_t policy, int flags)
int hwloc_set_proc_membind (hwloc_topology_t topology, hwloc_pid_t pid,
hwloc_const_cpuset_t cpuset, hwloc_membind_policy_t policy, int
flags)
int hwloc_get_proc_membind_nodeset (hwloc_topology_t topology,
hwloc_pid_t pid, hwloc_nodeset_t nodeset, hwloc_membind_policy_t
*policy, int flags)
int hwloc_get_proc_membind (hwloc_topology_t topology, hwloc_pid_t pid,
hwloc_cpuset_t cpuset, hwloc_membind_policy_t *policy, int flags)
int hwloc_set_area_membind_nodeset (hwloc_topology_t topology, const
void *addr, size_t len, hwloc_const_nodeset_t nodeset,
hwloc_membind_policy_t policy, int flags)
int hwloc_set_area_membind (hwloc_topology_t topology, const void
*addr, size_t len, hwloc_const_cpuset_t cpuset,
hwloc_membind_policy_t policy, int flags)
int hwloc_get_area_membind_nodeset (hwloc_topology_t topology, const
void *addr, size_t len, hwloc_nodeset_t nodeset,
hwloc_membind_policy_t *policy, int flags)
int hwloc_get_area_membind (hwloc_topology_t topology, const void
*addr, size_t len, hwloc_cpuset_t cpuset, hwloc_membind_policy_t
*policy, int flags)
void * hwloc_alloc (hwloc_topology_t topology, size_t len)
void * hwloc_alloc_membind_nodeset (hwloc_topology_t topology, size_t
len, hwloc_const_nodeset_t nodeset, hwloc_membind_policy_t policy,
int flags)
void * hwloc_alloc_membind (hwloc_topology_t topology, size_t len,
hwloc_const_cpuset_t cpuset, hwloc_membind_policy_t policy, int
flags)
static void * hwloc_alloc_membind_policy_nodeset (hwloc_topology_t
topology, size_t len, hwloc_const_nodeset_t nodeset,
hwloc_membind_policy_t policy, int flags)
static void * hwloc_alloc_membind_policy (hwloc_topology_t topology,
size_t len, hwloc_const_cpuset_t set, hwloc_membind_policy_t
policy, int flags)
int hwloc_free (hwloc_topology_t topology, void *addr, size_t len)
Detailed Description
Memory binding can be done three ways:
· explicit memory allocation thanks to hwloc_alloc_membind() and
friends: the binding will have effect on the memory allocated by
these functions.
· implicit memory binding through binding policy: hwloc_set_membind()
and friends only define the current policy of the process, which will
be applied to the subsequent calls to malloc() and friends.
· migration of existing memory ranges, thanks to
hwloc_set_area_membind() and friends, which move already-allocated
data.
Not all operating systems support all three ways.
hwloc_topology_get_support() may be used to query about the actual
memory binding support in the currently used operating system.
When the requested binding operation is not available and the
HWLOC_MEMBIND_STRICT flag was passed, the function returns -1. errno
will be set to ENOSYS when the system does support the specified action
or policy (e.g., some systems only allow binding memory on a per-thread
basis, whereas other systems only allow binding memory for all threads
in a process). errno will be set to EXDEV when the requested cpuset can
not be enforced (e.g., some systems only allow binding memory to a
single NUMA node).
If HWLOC_MEMBIND_STRICT was not passed, the function may fail as well,
or the operating system may use a slightly different operation (with
side-effects, smaller binding set, etc.) when the requested operation
is not exactly supported.
The most portable form that should be preferred over the others
whenever possible is as follows. It allocates some memory hopefully
bound to the specified set. To do so, hwloc will possibly have to
change the current memory binding policy in order to actually get the
memory bound, if the OS does not provide any other way to simply
allocate bound memory without changing the policy for all allocations.
That is the difference with hwloc_alloc_membind(), which will never
change the current memory binding policy.
hwloc_alloc_membind_policy(topology, size, set,
HWLOC_MEMBIND_BIND, 0);
Each hwloc memory binding function is available in two forms: one that
takes a CPU set argument and another that takes a NUMA memory node set
argument (see Object Sets (hwloc_cpuset_t and hwloc_nodeset_t) and The
bitmap API for a discussion of CPU sets and NUMA memory node sets). The
names of the latter form end with _nodeset. It is also possible to
convert between CPU set and node set using hwloc_cpuset_to_nodeset() or
hwloc_cpuset_from_nodeset().
See also:
Some example codes are available under doc/examples/ in the source
tree.
Note:
On some operating systems, memory binding affects the CPU binding;
see HWLOC_MEMBIND_NOCPUBIND
Enumeration Type Documentation
enum hwloc_membind_flags_t
Memory binding flags. These flags can be used to refine the binding
policy. All flags can be logically OR'ed together with the exception of
HWLOC_MEMBIND_PROCESS and HWLOC_MEMBIND_THREAD; these two flags are
mutually exclusive.
Not all systems support all kinds of binding.
hwloc_topology_get_support() may be used to query about the actual
memory binding support in the currently used operating system. See the
'Detailed Description' section of Memory binding for a description of
errors that can occur.
Enumerator
HWLOC_MEMBIND_PROCESS
Set policy for all threads of the specified (possibly
multithreaded) process. This flag is mutually exclusive with
HWLOC_MEMBIND_THREAD.
HWLOC_MEMBIND_THREAD
Set policy for a specific thread of the current process. This
flag is mutually exclusive with HWLOC_MEMBIND_PROCESS.
HWLOC_MEMBIND_STRICT
Request strict binding from the OS. The function will fail if
the binding can not be guaranteed / completely enforced.
This flag has slightly different meanings depending on which function
it is used with.
HWLOC_MEMBIND_MIGRATE
Migrate existing allocated memory. If the memory cannot be
migrated and the HWLOC_MEMBIND_STRICT flag is passed, an error
will be returned.
HWLOC_MEMBIND_NOCPUBIND
Avoid any effect on CPU binding. On some operating systems, some
underlying memory binding functions also bind the application to
the corresponding CPU(s). Using this flag will cause hwloc to
avoid using OS functions that could potentially affect CPU
bindings. Note, however, that using NOCPUBIND may reduce hwloc's
overall memory binding support. Specifically: some of hwloc's
memory binding functions may fail with errno set to ENOSYS when
used with NOCPUBIND.
enum hwloc_membind_policy_t
Memory binding policy. These constants can be used to choose the
binding policy. Only one policy can be used at a time (i.e., the values
cannot be OR'ed together).
Not all systems support all kinds of binding.
hwloc_topology_get_support() may be used to query about the actual
memory binding policy support in the currently used operating system.
See the 'Detailed Description' section of Memory binding for a
description of errors that can occur.
Enumerator
HWLOC_MEMBIND_DEFAULT
Reset the memory allocation policy to the system default.
Depending on the operating system, this may correspond to
HWLOC_MEMBIND_FIRSTTOUCH (Linux), or HWLOC_MEMBIND_BIND (AIX,
HP-UX, OSF, Solaris, Windows).
HWLOC_MEMBIND_FIRSTTOUCH
Allocate memory but do not immediately bind it to a specific
locality. Instead, each page in the allocation is bound only
when it is first touched. Pages are individually bound to the
local NUMA node of the first thread that touches it. If there is
not enough memory on the node, allocation may be done in the
specified cpuset before allocating on other nodes.
HWLOC_MEMBIND_BIND
Allocate memory on the specified nodes.
HWLOC_MEMBIND_INTERLEAVE
Allocate memory on the given nodes in an interleaved / round-
robin manner. The precise layout of the memory across multiple
NUMA nodes is OS/system specific. Interleaving can be useful
when threads distributed across the specified NUMA nodes will
all be accessing the whole memory range concurrently, since the
interleave will then balance the memory references.
HWLOC_MEMBIND_REPLICATE
Replicate memory on the given nodes; reads from this memory will
attempt to be serviced from the NUMA node local to the reading
thread. Replicating can be useful when multiple threads from the
specified NUMA nodes will be sharing the same read-only data.
This policy can only be used with existing memory allocations
(i.e., the hwloc_set_*membind*() functions); it cannot be used
with functions that allocate new memory (i.e., the
hwloc_alloc*() functions).
HWLOC_MEMBIND_NEXTTOUCH
For each page bound with this policy, by next time it is touched
(and next time only), it is moved from its current location to
the local NUMA node of the thread where the memory reference
occurred (if it needs to be moved at all).
HWLOC_MEMBIND_MIXED
Returned by get_membind() functions when multiple threads or
parts of a memory area have differing memory binding policies.
Function Documentation
void* hwloc_alloc (hwloc_topology_t topology, size_t len)
Allocate some memory. This is equivalent to malloc(), except that it
tries to allocate page-aligned memory from the OS.
Note:
The allocated memory should be freed with hwloc_free().
void* hwloc_alloc_membind (hwloc_topology_t topology, size_t len,
hwloc_const_cpuset_t cpuset, hwloc_membind_policy_t policy, int flags)
Allocate some memory on memory nodes near the given physical cpuset
cpuset.
Returns:
NULL with errno set to ENOSYS if the action is not supported and
HWLOC_MEMBIND_STRICT is given
NULL with errno set to EXDEV if the binding cannot be enforced and
HWLOC_MEMBIND_STRICT is given
NULL with errno set to ENOMEM if the memory allocation failed even
before trying to bind.
Note:
The allocated memory should be freed with hwloc_free().
void* hwloc_alloc_membind_nodeset (hwloc_topology_t topology, size_t len,
hwloc_const_nodeset_t nodeset, hwloc_membind_policy_t policy, int
flags)
Allocate some memory on the given physical nodeset nodeset.
Returns:
NULL with errno set to ENOSYS if the action is not supported and
HWLOC_MEMBIND_STRICT is given
NULL with errno set to EXDEV if the binding cannot be enforced and
HWLOC_MEMBIND_STRICT is given
NULL with errno set to ENOMEM if the memory allocation failed even
before trying to bind.
Note:
The allocated memory should be freed with hwloc_free().
static void* hwloc_alloc_membind_policy (hwloc_topology_t topology, size_t
len, hwloc_const_cpuset_t set, hwloc_membind_policy_t policy, int
flags) [inline], [static]
Allocate some memory on the memory nodes near given cpuset cpuset. This
is similar to hwloc_alloc_membind_policy_nodeset(), but for a given
cpuset.
static void* hwloc_alloc_membind_policy_nodeset (hwloc_topology_t topology,
size_t len, hwloc_const_nodeset_t nodeset, hwloc_membind_policy_t
policy, int flags) [inline], [static]
Allocate some memory on the given nodeset nodeset. This is similar to
hwloc_alloc_membind() except that it is allowed to change the current
memory binding policy, thus providing more binding support, at the
expense of changing the current state.
int hwloc_free (hwloc_topology_t topology, void * addr, size_t len)
Free memory that was previously allocated by hwloc_alloc() or
hwloc_alloc_membind().
int hwloc_get_area_membind (hwloc_topology_t topology, const void * addr,
size_t len, hwloc_cpuset_t cpuset, hwloc_membind_policy_t * policy, int
flags)
Query the CPUs near the physical NUMA node(s) and binding policy of the
memory identified by (addr, len ). This function has two output
parameters: cpuset and policy. The values returned in these parameters
depend on both the flags passed in and the memory binding policies and
nodesets of the pages in the address range.
If HWLOC_MEMBIND_STRICT is specified, the target pages are first
checked to see if they all have the same memory binding policy and
nodeset. If they do not, -1 is returned and errno is set to EXDEV. If
they are identical across all pages, the policy is returned in policy.
cpuset is set to the union of CPUs near the NUMA node(s) in the
nodeset.
If HWLOC_MEMBIND_STRICT is not specified, the union of all NUMA node(s)
containing pages in the address range is calculated. cpuset is then set
to the CPUs near the NUMA node(s) in this union. If all pages in the
target have the same policy, it is returned in policy. Otherwise,
policy is set to HWLOC_MEMBIND_MIXED.
If any other flags are specified, -1 is returned and errno is set to
EINVAL.
int hwloc_get_area_membind_nodeset (hwloc_topology_t topology, const void *
addr, size_t len, hwloc_nodeset_t nodeset, hwloc_membind_policy_t *
policy, int flags)
Query the physical NUMA node(s) and binding policy of the memory
identified by (addr, len ). This function has two output parameters:
nodeset and policy. The values returned in these parameters depend on
both the flags passed in and the memory binding policies and nodesets
of the pages in the address range.
If HWLOC_MEMBIND_STRICT is specified, the target pages are first
checked to see if they all have the same memory binding policy and
nodeset. If they do not, -1 is returned and errno is set to EXDEV. If
they are identical across all pages, the nodeset and policy are
returned in nodeset and policy, respectively.
If HWLOC_MEMBIND_STRICT is not specified, nodeset is set to the union
of all NUMA node(s) containing pages in the address range. If all pages
in the target have the same policy, it is returned in policy.
Otherwise, policy is set to HWLOC_MEMBIND_MIXED.
If any other flags are specified, -1 is returned and errno is set to
EINVAL.
int hwloc_get_membind (hwloc_topology_t topology, hwloc_cpuset_t cpuset,
hwloc_membind_policy_t * policy, int flags)
Query the default memory binding policy and physical locality of the
current process or thread (the locality is returned in cpuset as CPUs
near the locality's actual NUMA node(s)). This function has two output
parameters: cpuset and policy. The values returned in these parameters
depend on both the flags passed in and the current memory binding
policies and nodesets in the queried target.
Passing the HWLOC_MEMBIND_PROCESS flag specifies that the query target
is the current policies and nodesets for all the threads in the current
process. Passing HWLOC_MEMBIND_THREAD specifies that the query target
is the current policy and nodeset for only the thread invoking this
function.
If neither of these flags are passed (which is the most portable
method), the process is assumed to be single threaded. This allows
hwloc to use either process-based OS functions or thread-based OS
functions, depending on which are available.
HWLOC_MEMBIND_STRICT is only meaningful when HWLOC_MEMBIND_PROCESS is
also specified. In this case, hwloc will check the default memory
policies and nodesets for all threads in the process. If they are not
identical, -1 is returned and errno is set to EXDEV. If they are
identical, the policy is returned in policy. cpuset is set to the union
of CPUs near the NUMA node(s) in the nodeset.
Otherwise, if HWLOC_MEMBIND_PROCESS is specified (and
HWLOC_MEMBIND_STRICT is not specified), the default nodeset from each
thread is logically OR'ed together. cpuset is set to the union of CPUs
near the NUMA node(s) in the resulting nodeset. If all threads' default
policies are the same, policy is set to that policy. If they are
different, policy is set to HWLOC_MEMBIND_MIXED.
In the HWLOC_MEMBIND_THREAD case (or when neither HWLOC_MEMBIND_PROCESS
or HWLOC_MEMBIND_THREAD is specified), there is only one nodeset and
policy. The policy is returned in policy; cpuset is set to the union of
CPUs near the NUMA node(s) in the nodeset.
If any other flags are specified, -1 is returned and errno is set to
EINVAL.
int hwloc_get_membind_nodeset (hwloc_topology_t topology, hwloc_nodeset_t
nodeset, hwloc_membind_policy_t * policy, int flags)
Query the default memory binding policy and physical locality of the
current process or thread. This function has two output parameters:
nodeset and policy. The values returned in these parameters depend on
both the flags passed in and the current memory binding policies and
nodesets in the queried target.
Passing the HWLOC_MEMBIND_PROCESS flag specifies that the query target
is the current policies and nodesets for all the threads in the current
process. Passing HWLOC_MEMBIND_THREAD specifies that the query target
is the current policy and nodeset for only the thread invoking this
function.
If neither of these flags are passed (which is the most portable
method), the process is assumed to be single threaded. This allows
hwloc to use either process-based OS functions or thread-based OS
functions, depending on which are available.
HWLOC_MEMBIND_STRICT is only meaningful when HWLOC_MEMBIND_PROCESS is
also specified. In this case, hwloc will check the default memory
policies and nodesets for all threads in the process. If they are not
identical, -1 is returned and errno is set to EXDEV. If they are
identical, the values are returned in nodeset and policy.
Otherwise, if HWLOC_MEMBIND_PROCESS is specified (and
HWLOC_MEMBIND_STRICT is not specified), nodeset is set to the logical
OR of all threads' default nodeset. If all threads' default policies
are the same, policy is set to that policy. If they are different,
policy is set to HWLOC_MEMBIND_MIXED.
In the HWLOC_MEMBIND_THREAD case (or when neither HWLOC_MEMBIND_PROCESS
or HWLOC_MEMBIND_THREAD is specified), there is only one nodeset and
policy; they are returned in nodeset and policy, respectively.
If any other flags are specified, -1 is returned and errno is set to
EINVAL.
int hwloc_get_proc_membind (hwloc_topology_t topology, hwloc_pid_t pid,
hwloc_cpuset_t cpuset, hwloc_membind_policy_t * policy, int flags)
Query the default memory binding policy and physical locality of the
specified process (the locality is returned in cpuset as CPUs near the
locality's actual NUMA node(s)). This function has two output
parameters: cpuset and policy. The values returned in these parameters
depend on both the flags passed in and the current memory binding
policies and nodesets in the queried target.
Passing the HWLOC_MEMBIND_PROCESS flag specifies that the query target
is the current policies and nodesets for all the threads in the
specified process. If HWLOC_MEMBIND_PROCESS is not specified (which is
the most portable method), the process is assumed to be single
threaded. This allows hwloc to use either process-based OS functions or
thread-based OS functions, depending on which are available.
Note that it does not make sense to pass HWLOC_MEMBIND_THREAD to this
function.
If HWLOC_MEMBIND_STRICT is specified, hwloc will check the default
memory policies and nodesets for all threads in the specified process.
If they are not identical, -1 is returned and errno is set to EXDEV. If
they are identical, the policy is returned in policy. cpuset is set to
the union of CPUs near the NUMA node(s) in the nodeset.
Otherwise, the default nodeset from each thread is logically OR'ed
together. cpuset is set to the union of CPUs near the NUMA node(s) in
the resulting nodeset. If all threads' default policies are the same,
policy is set to that policy. If they are different, policy is set to
HWLOC_MEMBIND_MIXED.
If any other flags are specified, -1 is returned and errno is set to
EINVAL.
Note:
hwloc_pid_t is pid_t on Unix platforms, and HANDLE on native
Windows platforms.
int hwloc_get_proc_membind_nodeset (hwloc_topology_t topology, hwloc_pid_t
pid, hwloc_nodeset_t nodeset, hwloc_membind_policy_t * policy, int
flags)
Query the default memory binding policy and physical locality of the
specified process. This function has two output parameters: nodeset and
policy. The values returned in these parameters depend on both the
flags passed in and the current memory binding policies and nodesets in
the queried target.
Passing the HWLOC_MEMBIND_PROCESS flag specifies that the query target
is the current policies and nodesets for all the threads in the
specified process. If HWLOC_MEMBIND_PROCESS is not specified (which is
the most portable method), the process is assumed to be single
threaded. This allows hwloc to use either process-based OS functions or
thread-based OS functions, depending on which are available.
Note that it does not make sense to pass HWLOC_MEMBIND_THREAD to this
function.
If HWLOC_MEMBIND_STRICT is specified, hwloc will check the default
memory policies and nodesets for all threads in the specified process.
If they are not identical, -1 is returned and errno is set to EXDEV. If
they are identical, the values are returned in nodeset and policy.
Otherwise, nodeset is set to the logical OR of all threads' default
nodeset. If all threads' default policies are the same, policy is set
to that policy. If they are different, policy is set to
HWLOC_MEMBIND_MIXED.
If any other flags are specified, -1 is returned and errno is set to
EINVAL.
Note:
hwloc_pid_t is pid_t on Unix platforms, and HANDLE on native
Windows platforms.
int hwloc_set_area_membind (hwloc_topology_t topology, const void * addr,
size_t len, hwloc_const_cpuset_t cpuset, hwloc_membind_policy_t policy,
int flags)
Bind the already-allocated memory identified by (addr, len) to the NUMA
node(s) near physical cpuset.
Returns:
-1 with errno set to ENOSYS if the action is not supported
-1 with errno set to EXDEV if the binding cannot be enforced
int hwloc_set_area_membind_nodeset (hwloc_topology_t topology, const void *
addr, size_t len, hwloc_const_nodeset_t nodeset, hwloc_membind_policy_t
policy, int flags)
Bind the already-allocated memory identified by (addr, len) to the NUMA
node(s) in physical nodeset.
Returns:
-1 with errno set to ENOSYS if the action is not supported
-1 with errno set to EXDEV if the binding cannot be enforced
int hwloc_set_membind (hwloc_topology_t topology, hwloc_const_cpuset_t
cpuset, hwloc_membind_policy_t policy, int flags)
Set the default memory binding policy of the current process or thread
to prefer the NUMA node(s) near the specified physical cpuset. If
neither HWLOC_MEMBIND_PROCESS nor HWLOC_MEMBIND_THREAD is specified,
the current process is assumed to be single-threaded. This is the most
portable form as it permits hwloc to use either process-based OS
functions or thread-based OS functions, depending on which are
available.
Returns:
-1 with errno set to ENOSYS if the action is not supported
-1 with errno set to EXDEV if the binding cannot be enforced
int hwloc_set_membind_nodeset (hwloc_topology_t topology,
hwloc_const_nodeset_t nodeset, hwloc_membind_policy_t policy, int
flags)
Set the default memory binding policy of the current process or thread
to prefer the NUMA node(s) specified by physical nodeset. If neither
HWLOC_MEMBIND_PROCESS nor HWLOC_MEMBIND_THREAD is specified, the
current process is assumed to be single-threaded. This is the most
portable form as it permits hwloc to use either process-based OS
functions or thread-based OS functions, depending on which are
available.
Returns:
-1 with errno set to ENOSYS if the action is not supported
-1 with errno set to EXDEV if the binding cannot be enforced
int hwloc_set_proc_membind (hwloc_topology_t topology, hwloc_pid_t pid,
hwloc_const_cpuset_t cpuset, hwloc_membind_policy_t policy, int flags)
Set the default memory binding policy of the specified process to
prefer the NUMA node(s) near the specified physical cpuset.
Returns:
-1 with errno set to ENOSYS if the action is not supported
-1 with errno set to EXDEV if the binding cannot be enforced
Note:
hwloc_pid_t is pid_t on Unix platforms, and HANDLE on native
Windows platforms.
int hwloc_set_proc_membind_nodeset (hwloc_topology_t topology, hwloc_pid_t
pid, hwloc_const_nodeset_t nodeset, hwloc_membind_policy_t policy, int
flags)
Set the default memory binding policy of the specified process to
prefer the NUMA node(s) specified by physical nodeset.
Returns:
-1 with errno set to ENOSYS if the action is not supported
-1 with errno set to EXDEV if the binding cannot be enforced
Note:
hwloc_pid_t is pid_t on Unix platforms, and HANDLE on native
Windows platforms.
Author
Generated automatically by Doxygen for Hardware Locality (hwloc) from
the source code.
Version 1.11.1 Thu Oct 15 2015 hwlocality_membinding(3)
