Sun Fire[TM] Servers: Memory Placement Optimization (MPO) Frequently Asked Questions (FAQ)

Asset ID:	1-71-1012194.1
Update Date:	2011-01-12
Keywords:

Solution Type Technical Instruction Sure

Solution 1012194.1 : Sun Fire[TM] Servers: Memory Placement Optimization (MPO) Frequently Asked Questions (FAQ)

Applies to:

Sun Fire 3800 Server - Version: Not Applicable and later   [Release: N/A and later ]
Sun Fire 4800 Server - Version: Not Applicable and later    [Release: N/A and later]
Sun Fire 4810 Server - Version: Not Applicable and later    [Release: N/A and later]
Sun Fire 6800 Server - Version: Not Applicable and later    [Release: N/A and later]
Sun Fire 12K Server - Version: Not Applicable and later    [Release: N/A and later]
All Platforms

Goal

This document provides information on some frequently asked questions about MPO (Memory Placement Optimization) on Sun Fire servers.

Questions included in this document are:

Q1: What is MPO?
Q2: Where can one find out more about MPO?
Q3: How can one disable MPO?
Q4: Why is the number of latency groups (nlgrps) larger than the number of system boards on the system?
Q5: How do I know if MPO is enabled ?
Q6: How do I know if non null proc LPA is enabled ?

Solution

Q1: What is MPO?

A1: MPO is the abbreviation for Memory Placement Optimization. MPO is a kernel implementation of an aspect of Non Uniform Memory Architecture (NUMA) for the Sun Fire[TM] line of systems. Introduced in the Solaris[TM] 9 9/02 Operating System, MPO optimizes the scheduling of light weight processes to reduce memory latency and improve application performance.

Q2: Where can one find out more about MPO?

A2: You can find a white paper that describes MPO attached to this article and linked here (pdf).

NOTE: This white paper used to be located at http://www.sun.com/servers/wp/docs/mpo_v7_CUSTOMER.pdf a link which no longer works.

Q3: How can one disable MPO?

A3: Add this entry to the /etc/system file, and reboot the system:

set lgrp_mem_default_policy=4

The "lgrp_mem_default_policy" variable is a kernel variable that specifies the default memory allocation policy. Memory allocation policies are used to control how memory is allocated under MPO. The basic policies are "first/next touch" and "random." The "first/next touch policy" allocates memory from the home lgroup of the thread that first touches the memory. The "random memory allocation policy" allocates memory randomly from lgroups in the system. To get the full benefit of MPO, set the lgrp_mem_default_policy to one (1) for next touch.

Q4: Why is the number of latency groups (nlgrps) larger than the number of system boards on the system?

A4: For MPO to work, the number of nlgrps should be greater than a value of one (1). There is always one lgrp (root lgrp) even on desktop systems, such as the Ultra[TM] 5 or Ultra 10, reserved for root lgrp.
"Root lgrp" represents the set of resources at the system-wide level of locality. When there is only one system board, nlgrp=2, lgrp optimization is disabled; that is, the kernel does not perform page migration.

Q5: How do I know if MPO is enabled ?

A5: Perform the following:
1) The number of lgroups is greater than 1:

# echo nlgrps/X | mdb -k
nlgrps:
nlgrps:        3

2) The default memory allocation policy is set to the "next touch policy". Verify this by checking that lgrp_mem_default_policy is "1":

# echo lgrp_mem_default_policy/X | mdb -k
lgrp_mem_default_policy:
lgrp_mem_default_policy:        1

Q6: How do I know if non null proc LPA is enabled ?

A6: Local Physical Address (LPA) identifies a physical range of memory local to a processor. The range is identified as a base and a bound which when both 0 (null LPA) means that the processor treats all coherent memory references as remote. # cfgadm -x passthru -o showlpa SB::cpu0 where is an appropriate system board number

Example :
Non-null-proc-lpa :

# cfgadm -x passthru -o showlpa SB2::cpu0
showlpa SB2::cpu0 portid 64, base pa 10000000000, bound pa 12000000000

null-proc-lpa :

# cfgadm -x passthru -o showlpa SB2::cpu0
showlpa SB2::cpu0 portid 64, base pa 0, bound pa

Internal Comments
More information available internally :
http://agares.central/numa/docs/tuneables.html
http://agares.central/numa/docs/mpo_lpa.html

Previously Published As 76905

Attachments

This solution has no attachment