Skip to content

mallocng ¤

Implements handling of musl's allocator mallocng. https://elixir.bootlin.com/musl/v1.2.5/source/src/malloc/mallocng

Classes:

  • SlotState
  • Group

    A group is an array of slots.

  • Slot

    The "unit of allocation" (analogous to glibc's "chunk").

  • GroupedSlot

    This is not a mallocng concept, this is a pwndbg abstraction.

  • Meta

    The metadata of a group.

  • MetaArea

    Slabs that contain metas, linked in a singly-linked list.

  • MallocContext

    The global object that holds all allocator state.

  • Mallocng

    Tracks the allocator state.

Functions:

Attributes:

UNIT module-attribute ¤

UNIT: int = 16

IB module-attribute ¤

IB: int = 4

size_classes module-attribute ¤

size_classes: list[int] = [
    1,
    2,
    3,
    4,
    5,
    6,
    7,
    8,
    9,
    10,
    12,
    15,
    18,
    20,
    25,
    31,
    36,
    42,
    50,
    63,
    72,
    84,
    102,
    127,
    146,
    170,
    204,
    255,
    292,
    340,
    409,
    511,
    584,
    682,
    818,
    1023,
    1169,
    1364,
    1637,
    2047,
    2340,
    2730,
    3276,
    4095,
    4680,
    5460,
    6552,
    8191,
]

mallocng module-attribute ¤

mallocng = Mallocng()

SlotState ¤

Bases: Enum

Attributes:

ALLOCATED class-attribute instance-attribute ¤

ALLOCATED = 'allocated'

FREED class-attribute instance-attribute ¤

FREED = 'freed'

AVAIL class-attribute instance-attribute ¤

AVAIL = 'available'

Group ¤

Group(addr: int)

A group is an array of slots.

https://elixir.bootlin.com/musl/v1.2.5/source/src/malloc/mallocng/meta.h#L17 struct group { struct meta *meta; unsigned char active_idx:5; char pad[UNIT - sizeof(struct meta *) - 1]; unsigned char storage[]; };

Methods:

  • preload

    Read all the necessary process memory to populate the group's

  • set_meta

    Sets the meta object for this group.

  • at_index

    Get the address of the slot at index idx.

Attributes:

addr instance-attribute ¤

addr = addr

meta property ¤

meta: Meta

Raises:

  • Error

    When reading memory fails.

active_idx property ¤

active_idx: int

Raises:

  • Error

    When reading memory fails.

storage property ¤

storage: int

group_size property ¤

group_size: int

The size of this group, in bytes.

Raises:

  • Error

    When reading meta fails.

preload ¤

preload() -> None

Read all the necessary process memory to populate the group's fields.

Do this if you know you will be using most of the fields of the group. It will be faster, since we can do one reads instead of two small ones. You may also catch inaccessible memory exceptions here and not worry about it later.

Raises:

  • Error

    When reading memory fails.

set_meta ¤

set_meta(meta: Meta) -> None

Sets the meta object for this group.

If the meta for this group is already calculated by the callee, use this to prevent it from being wastefully recalculated.

at_index ¤

at_index(idx: int) -> int

Get the address of the slot at index idx.

Slot ¤

Slot(p: int)

The "unit of allocation" (analogous to glibc's "chunk"). There is no struct in the source code that describes it.

Methods:

Attributes:

p instance-attribute ¤

p: int = p

offset property ¤

offset: int

Raises:

  • Error

    When reading memory fails.

pn3 property ¤

pn3: int

Raises:

  • Error

    When reading memory fails.

idx property ¤

idx: int

Raises:

  • Error

    When reading memory fails.

reserved_in_header property ¤

reserved_in_header: int

big_offset_check property ¤

big_offset_check: int

Raises:

  • Error

    When reading memory fails.

start property ¤

start: int

Raises:

  • Error

    When reading meta fails.

cyclic_offset property ¤

cyclic_offset: int

Returns zero if is_cyclic() is False.

Raises:

  • Error

    When reading meta fails.

startn3 property ¤

startn3: int

Raises:

  • Error

    When reading memory fails.

reserved_in_footer: int

Returns -1 if the value is invalid, i.e. reserved_in_header() != 5.

Raises:

  • Error

    When reading memory fails.

end property ¤

end: int

Raises:

  • Error

    When reading meta fails.

reserved property ¤

reserved: int

Returns 0 if reserved_in_header() == 6. Returns -1 if reserved_in_header() == 7.

Raises:

  • Error

    When reading memory fails.

nominal_size property ¤

nominal_size: int

Raises:

  • Error

    When reading meta fails.

user_size property ¤

user_size: int

Raises:

  • Error

    When reading meta fails.

slack property ¤

slack: int

Raises:

  • Error

    When reading meta fails.

group property ¤

group: Group

meta property ¤

meta: Meta

Raises:

  • Error

    When reading memory fails.

slot_state property ¤

slot_state: SlotState

preload ¤

preload() -> None

Read all the necessary process memory to populate the slot's p header fields.

Do this if you know you will be using most of the fields of the slot. It will be faster, since we can do a few big reads instead of many small ones. You may also catch inaccessible memory exceptions here and not worry about it later.

Fields dependant on the meta are not loaded - you will still need to worry about exceptions coming from them.

Raises:

  • Error

    When reading memory fails.

preload_meta_dependants ¤

preload_meta_dependants() -> None

Preloads all fields that depend on a sane meta.

It generally only makes sense to run this after preload(). Calling this reduces the amount of process writes and centralizes field exceptions to this function.

If both preload() and preload_meta_dependants() return without exceptions, all the fields in this class are guaranteed to not cause any more memory reads nor raise any more exceptions.

Raises:

  • Error

    When the meta is corrupt and/or reading memory fails.

is_cyclic ¤

is_cyclic() -> int

Returns whether mallocng reports that p != start.

contains_group ¤

contains_group() -> bool

Does this slot nest a group?

set_group ¤

set_group(group: Group) -> None

If the slot is FREED or AVAIL, it is impossible for it to recover the start of its group, and ergo its meta.

You can thus use this to set it externally.

from_p classmethod ¤

from_p(p: int) -> 'Slot'

from_start classmethod ¤

from_start(start: int) -> 'Slot'

GroupedSlot ¤

GroupedSlot(group: Group, idx: int)

This is not a mallocng concept, this is a pwndbg abstraction.

A Slot object uses its inband metadata to recover all its fields and uncover more information about itself by locating its group and meta. It works essentially the same way mallocng's free() works.

However, if a slot is freed or available, most of its in-band metadata will be invalid and it will not be able to recover group and meta. But, given the start of the slot, we can infer which group it belongs to and what its index is by walking allocator state i.e. ctx i.e. by using Mallocng.find_slot().

A GroupedSlot then describes all information we can glean about a slot which is described by a (group, idx) pair. Many of its fields can be completely different from a Slot at the same location. They are guaranteed to be the same only if the slot is ALLOCATED and hasn't been corrupted.

Not all fields that are available in Slot are available in GroupedSlot.

Make sure the group you are passing to the constructor points to a valid meta object.

Attributes:

group instance-attribute ¤

group = group

meta instance-attribute ¤

meta = meta

idx instance-attribute ¤

idx = idx

stride instance-attribute ¤

stride = stride

slot_state instance-attribute ¤

slot_state = slotstate_at_index(idx)

start instance-attribute ¤

start = storage + stride * idx

end instance-attribute ¤

end = start + stride - IB

Meta ¤

Meta(addr: int)

The metadata of a group.

https://elixir.bootlin.com/musl/v1.2.5/source/src/malloc/mallocng/meta.h#L24 struct meta { struct meta *prev, *next; struct group *mem; volatile int avail_mask, freed_mask; uintptr_t last_idx:5; uintptr_t freeable:1; uintptr_t sizeclass:6; uintptr_t maplen:8*sizeof(uintptr_t)-12; };

Methods:

  • preload

    Read all the necessary process memory to populate the meta's

  • parent_group

    If this group is nested, returns the address of the group which

  • root_group

    Returns the topmost/biggest parent group. It will never be a nested

  • slotstate_at_index
  • sizeof

Attributes:

  • addr (int) –
  • prev (int) –

    Raises:

  • next (int) –

    Raises:

  • mem (int) –

    Raises:

  • avail_mask (int) –

    Raises:

  • freed_mask (int) –

    Raises:

  • last_idx (int) –

    Raises:

  • freeable (int) –

    Raises:

  • sizeclass (int) –

    Raises:

  • maplen (int) –

    Raises:

  • stride (int) –

    Returns -1 if sizeclass >= len(size_classes).

  • cnt (int) –

    Number of slots in the group.

  • is_donated (bool) –

    Returns whether the group object referred to by this meta has been

  • is_mmaped (bool) –

    Returns whether the group object referred to by this meta has been

  • is_nested (bool) –

    Returns whether the group object referred to by this meta has been

addr instance-attribute ¤

addr: int = addr

prev property ¤

prev: int

Raises:

  • Error

    When reading memory fails.

next property ¤

next: int

Raises:

  • Error

    When reading memory fails.

mem property ¤

mem: int

Raises:

  • Error

    When reading memory fails.

avail_mask property ¤

avail_mask: int

Raises:

  • Error

    When reading memory fails.

freed_mask property ¤

freed_mask: int

Raises:

  • Error

    When reading memory fails.

last_idx property ¤

last_idx: int

Raises:

  • Error

    When reading memory fails.

freeable property ¤

freeable: int

Raises:

  • Error

    When reading memory fails.

sizeclass property ¤

sizeclass: int

Raises:

  • Error

    When reading memory fails.

maplen property ¤

maplen: int

Raises:

  • Error

    When reading memory fails.

stride property ¤

stride: int

Returns -1 if sizeclass >= len(size_classes).

cnt property ¤

cnt: int

Number of slots in the group.

is_donated property ¤

is_donated: bool

Returns whether the group object referred to by this meta has been created by being donated by ld.

is_mmaped property ¤

is_mmaped: bool

Returns whether the group object referred to by this meta has been created by being mmaped.

is_nested property ¤

is_nested: bool

Returns whether the group object referred to by this meta has been created by being nested into a slot.

preload ¤

preload() -> None

Read all the necessary process memory to populate the meta's fields.

Do this if you know you will be using most of the fields of the meta. It will be faster, since we can do a one big read instead of many small ones. You may also catch inaccessible memory exceptions here and not worry about it later.

Raises:

  • Error

    When reading memory fails.

parent_group ¤

parent_group() -> int

If this group is nested, returns the address of the group which contains the slot in which this group is in. Otherwise, returns -1.

root_group ¤

root_group() -> Group

Returns the topmost/biggest parent group. It will never be a nested group. If this group isn't nested, this group is returned.

slotstate_at_index ¤

slotstate_at_index(idx: int) -> SlotState

sizeof staticmethod ¤

sizeof() -> int

MetaArea ¤

MetaArea(addr: int)

Slabs that contain metas, linked in a singly-linked list.

https://elixir.bootlin.com/musl/v1.2.5/source/src/malloc/mallocng/meta.h#L34 struct meta_area { uint64_t check; struct meta_area *next; int nslots; struct meta slots[]; };

Methods:

  • load
  • at_index

    Returns the address of the meta object located

Attributes:

addr instance-attribute ¤

addr: int = addr

check instance-attribute ¤

check: int = 0

meta_area instance-attribute ¤

meta_area: int = 0

nslots instance-attribute ¤

nslots: int = 0

slots instance-attribute ¤

slots: int = 0

area_size property ¤

area_size: int

Returns not the size of struct meta_area but rather the size of the memory this object represents.

load ¤

load() -> None

at_index ¤

at_index(idx: int) -> int

Returns the address of the meta object located at index idx.

MallocContext ¤

MallocContext(addr: int)

The global object that holds all allocator state.

https://elixir.bootlin.com/musl/v1.2.5/source/src/malloc/mallocng/meta.h#L41 struct malloc_context { uint64_t secret;

ifndef PAGESIZE¤

size_t pagesize;

endif¤

int init_done; unsigned mmap_counter; struct meta *free_meta_head; struct meta *avail_meta; size_t avail_meta_count, avail_meta_area_count, meta_alloc_shift; struct meta_area *meta_area_head, *meta_area_tail; unsigned char *avail_meta_areas; struct meta *active[48]; size_t usage_by_class[48]; uint8_t unmap_seq[32], bounces[32]; uint8_t seq; uintptr_t brk; };

Methods:

  • load
  • looks_valid

    Returns true if this object looks like a valid struct malloc_context object

Attributes:

addr instance-attribute ¤

addr: int = addr

secret instance-attribute ¤

secret: int = 0

pagesize instance-attribute ¤

pagesize: int = 0

init_done instance-attribute ¤

init_done: int = 0

mmap_counter instance-attribute ¤

mmap_counter: int = 0

free_meta_head instance-attribute ¤

free_meta_head: int = 0

avail_meta instance-attribute ¤

avail_meta: int = 0

avail_meta_count instance-attribute ¤

avail_meta_count: int = 0

avail_meta_area_count instance-attribute ¤

avail_meta_area_count: int = 0

meta_alloc_shift instance-attribute ¤

meta_alloc_shift: int = 0

meta_area_head instance-attribute ¤

meta_area_head: int = 0

meta_area_tail instance-attribute ¤

meta_area_tail: int = 0

avail_meta_areas instance-attribute ¤

avail_meta_areas: int = 0

active instance-attribute ¤

active: list[int] = []

usage_by_class instance-attribute ¤

usage_by_class: list[int] = []

unmap_seq instance-attribute ¤

unmap_seq: list[int] = []

bounces instance-attribute ¤

bounces: list[int] = []

seq instance-attribute ¤

seq: int = 0

brk instance-attribute ¤

brk: int = 0

sizeof instance-attribute ¤

sizeof: int = 0

has_pagesize_field instance-attribute ¤

has_pagesize_field: bool = False

load ¤

load() -> None

looks_valid ¤

looks_valid() -> bool

Returns true if this object looks like a valid struct malloc_context object describing an initialized heap. False otherwise.

This is used by class Mallocng to find the correct ctx object.

We consider it invalid if the heap reads as uninitialized because: 1. Performing this check filters out invalid ctx objects very well. 2. When musl is dynmically linked, due to the ld donation logic, the heap will usually be initialized before the start of main().

Mallocng ¤

Mallocng()

Bases: MemoryAllocator

Tracks the allocator state. By leveraging the __malloc_context symbol.

Import this singleton class like: from pwndbg.aglib.heap.mallocng import mallocng as ng

and make sure that you have run ng.init_if_needed() before you used the object.

Methods:

  • init_if_needed

    We want this class to be a singleton, but also we can't

  • set_ctx_addr

    Find where the __malloc_context global symbol is. Try using debug information,

  • libc_has_debug_syms
  • find_slot

    Get the slot which contains this address.

  • containing

    Same as find_slot() but returns only the start address of the slot, or zero

  • get_free_metas

    Get all free metas by traversing the ctx.free_meta_head doubly

  • meta_is_avail

    Checks whether a meta is available.

  • is_initialized

    Returns whether the allocator is initialized or not.

Attributes:

finished_init instance-attribute ¤

finished_init: bool = False

ctx_addr instance-attribute ¤

ctx_addr: int = 0

ctx instance-attribute ¤

ctx: MallocContext | None = None

has_debug_syms instance-attribute ¤

has_debug_syms: bool = False

init_if_needed ¤

init_if_needed() -> bool

We want this class to be a singleton, but also we can't initialize it as soon as pwndbg is loaded.

Users of the object are responsible for calling this to make sure the object is initialized. This also ensures our view of the heap is up-to-date.

Returns:

  • bool

    True if this object is successfully initialized (whether

  • bool

    now or before). False otherswise. If this returns False

  • bool

    you may not use this object for heap operations.

set_ctx_addr ¤

set_ctx_addr() -> None

Find where the __malloc_context global symbol is. Try using debug information, but if it isn't available try using a heuristic.

libc_has_debug_syms ¤

libc_has_debug_syms() -> bool

find_slot ¤

find_slot(
    address: int, metadata: bool = False, shallow: bool = False
) -> tuple[GroupedSlot | None, Slot | None]

Get the slot which contains this address.

We say a slot "contains" an address, if the address is in [start, start + stride). Thus, this will match the previous slot if you provide the address of the header inband metadata of a slot.

If metadata is True, then we check [start - IB, end) for containment.

If shallow is True, return the biggest slot which contains this address. The group that owns this slot will not be a nested group.

Returns (None, None) if nothing is found.

containing ¤

containing(address: int, metadata: bool = False, shallow: bool = False) -> int

Same as find_slot() but returns only the start address of the slot, or zero if no slot is found.

get_free_metas ¤

get_free_metas() -> dict[int, tuple[int, Meta]]

Get all free metas by traversing the ctx.free_meta_head doubly linked list. Map them to their index in the list.

Raises:

  • Error

    If some meta cannot be read or is corrupted.

Returns:

  • dict[int, tuple[int, Meta]]

    A dictionary that maps: meta address -> (meta index in list, Meta object).

meta_is_avail ¤

meta_is_avail(addr: int) -> bool

Checks whether a meta is available.

is_initialized ¤

is_initialized() -> bool

Returns whether the allocator is initialized or not.

Returns:

  • bool

    A boolean.

int_size ¤

int_size() -> int