{"componentChunkName":"component---src-templates-post-template-js","path":"/ctf-angr-tutorial-en","result":{"data":{"markdownRemark":{"id":"501c6a9d-49d6-50d6-a0cc-04358e8ed50b","html":"
\n

This page has been machine-translated from the original page.

\n
\n

I had been using angr with template scripts without much thought, but recently I ran into a problem “where angr should work but doesn’t,” so I took it as a good opportunity to learn a bit more about it.

\n

When I actually used it properly, it turned out to be a far more powerful tool than I imagined, so I’m writing it up as an article along with some sample code — even if it only covers a fraction of what it can do.

\n

I plan to use this as a cheat sheet, so I’ll keep adding to it going forward.

\n\n

Table of Contents

\n\n

What is angr?

\n

angr is an analysis tool capable of performing various kinds of static analysis on binaries, including dynamic symbolic execution.

\n

angr can be used as a Python module.

\n

angr first loads a binary into a project and then instantiates several built-in classes.

\n

Reference: angr documentation

\n

When using angr in CTF, you typically use symbolic execution with symbolic variables to track how the program state changes, and leverage the internal SMT solver (z3, etc.) to reverse-compute the input value needed to produce a given output.

\n

Symbolic Execution

\n

angr performs analysis through symbolic execution using symbolic variables — not an emulator.

\n

The SimulationManager holds state (SimState) in units called stashes, and can step through and filter those states.

\n

During symbolic execution, angr fetches program instructions and applies them to produce a sequence of updated SimState objects.

\n

The initial stash is active. A SimState from which no further states can be derived after stepping is placed into the deadended stash.

\n

Using the explore() method of SimulationManager, you can search only for SimStates that reach a specific address and discard all others.

\n

You can also add constraints to symbolic variables (defined as Bitvectors) to reverse-compute the input that produces a given output.

\n

To solve for a Flag in a straightforward case, you can use a script like the following:

\n
import angr\n\nproj = angr.Project(\"chall\", auto_load_libs=False)\nobj = proj.loader.main_object\nprint(\"Entry\", hex(obj.entry))\n\nfind = 0x401654\navoids = [0x4016ca,0x4011a9]\n\ninit_state = proj.factory.entry_state()\nsimgr = proj.factory.simgr(init_state)\nsimgr.explore(find=find, avoid=avoids)\n\n# Output\nsimgr.found[0].posix.dumps(0)
\n

Below I summarize the classes and objects commonly used in CTF when performing symbolic execution with angr.

\n

The Factory Class

\n

Factory is one of the most frequently used classes in angr and is used to obtain code blocks and SimState objects for simulation.

\n

For example, you can retrieve a code block and read or modify registers and memory as follows:

\n
import angr\nproj = angr.Project(\"chall\", auto_load_libs=False)\n\n# Extract a basic code block\nblock = proj.factory.block(proj.entry)\nblock.pp()\n\n# Create a SimState object including the program's memory, registers, and filesystem\nstate = proj.factory.entry_state()\n\n# Get the register value at the current state\nstate.regs\nstate.regs.rax \n\n# Get the memory at the entry point\nstate.mem[proj.entry].int.resolved\n\n# Overwrite a register value with an arbitrary value\nstate.regs.rax = state.solver.BVV(3, 64)\n\n# Overwrite a memory location with an arbitrary value\nstate.mem[0x1000].long = 4
\n

Note: in angr, numbers are handled as bitvector objects.

\n

You can specify either 32-bit or 64-bit to match the CPU.

\n
# Convert an int to a 32-bit BVV\nbv = state.solver.BVV(0x1234, 32)\n\n# Convert a BVV to an int\nstate.solver.eval(bv)
\n

SimulationManager

\n

This is the object you use most often when solving CTF problems.

\n

SimulationManager lets you simulate program execution.

\n
# Create a simulation manager\nsimgr = proj.factory.simulation_manager(state)
\n

SimulationManager holds states in units called stashes.

\n

The initial stash given at initialization is active.

\n

When you perform symbolic execution with simgr.step(), you can see states being added to the active stash one by one.

\n

Therefore, after advancing execution for a while, inspecting elements in active shows that states have been updated.

\n

\n \n \n \n \n \n \n \n \n

\n

Analysis Techniques with SimulationManager

\n

Pre-defining the Flag String Length

\n

You can create a symbolic variable for the Flag with constraints as follows:

\n
import angr\nimport claripy\n\n# Create a symbolic variable 'flag' for 20 characters (8-bit bitvector)\nflag = claripy.BVS('flag', 20*8, explicit_name=True)\n\n# Create a SimState object including the program's memory, registers, and filesystem\n# state = proj.factory.blank_state(addr=funcaddr, add_options={angr.options.LAZY_SOLVES})\nstate = proj.factory.entry_state()\n\n# Add constraints to the 20-character symbolic variable\nfor i in range(19):\n    state.solver.add(flag.get_byte(i) >= 0x21)\n    state.solver.add(flag.get_byte(i) <= 0x7f)
\n

Adding Constraints on the Flag String

\n

If certain characters have already been identified, you can add constraints on those positions for more efficient exploration:

\n
# If a character at a specific position is already known, add it as a constraint\nstate.add_constraints(argv1.chop(8)[0] == 'C')\nstate.add_constraints(argv1.chop(8)[1] == 'T')\nstate.add_constraints(argv1.chop(8)[2] == 'F')\nstate.add_constraints(argv1.chop(8)[3] == '{')
\n

Applying the Flag Symbolic Variable at the Right Location

\n

The Flag symbolic variable you create must be applied at the appropriate location in accordance with the binary’s implementation.

\n
# When the Flag needs to be supplied as a command-line argument\nargv1 = claripy.BVS(\"argv1\", 0xE * 8)\nstate = proj.factory.entry_state(args=[\"./fairlight\", argv1])\n\n# When calling a function with the Flag as its argument\nflag = claripy.BVS('flag', 20*8, explicit_name=True)\nbuf = 0x606000 # buffer to store flag(.data)\nfuncaddr = 0x400646 # entry point of crazy function\n\nstate = proj.factory.blank_state(addr=funcaddr, add_options={angr.options.LAZY_SOLVES})\n\n# insert flag into memory by hand\n# state.memory.store(buf, flag, endness='Iend_LE')\nstate.memory.store(buf, flag, endness='Iend_BE')\nstate.regs.rdi = buf
\n

Improving Performance

\n

You can sometimes improve performance by automatically discarding stashes like deadended:

\n

Also, angr’s default maximum number of symbolic bytes that can be read from memory is 60 bytes; for longer Flags, analysis may not work correctly.

\n

In such cases you can specify the appropriate byte size by changing state.libc.buf_symbolic_bytes.

\n
# Disable automatic loading of external libraries to improve performance\nproj = angr.Project('./chall', auto_load_libs=False)\n\n# Specifying LAZY_SOLVES defers symbol resolution, which can improve performance (may reduce accuracy)\nstate = proj.factory.entry_state(args=[\"./chall\", argv1], add_options={angr.options.LAZY_SOLVES})\n\n# Change the maximum number of symbolic bytes read from memory\nstate.libc.buf_symbolic_bytes = input_size + 1\n\n# Automatically discard the avoid and deadended stashes\nsimgr.explore(find=find, avoid=avoids, step_func=lambda lsm: lsm.drop(stash='avoid').drop(stash='deadended'))
\n

Changing the Exploration Strategy

\n
simgr = proj.factory.simulation_manager(state)\nexplorer = angr.exploration_techniques.Explorer(find=lambda s: s.solver.eval(s.regs.dl) == 0x43 and s.solver.eval(s.regs.rip) == 0x401641)\nsimgr.use_technique(explorer)\nsimgr.run()
\n

Retrieving Register and Stack Information During Exploration

\n
# With a stash advanced to an arbitrary state via simgr.step() etc.\ncurrent_state = simgr.active[0]\n\n# Read the value of the RAX register\ncurrent_state.regs.rax\n\n# Get RSP information\ncurrent_state.regs.rsp\n\np_BV64 = current_state.mem[0x7fffffffffeffa8].uint64_t.resolved\ncurrent_state.solver.eval(p_BV64)\ncurrent_state.solver.eval_upto(p_BV64,10)
\n

Solving CTF Problems with angr

\n

Here I solve CTF problems introduced in the official angr documentation using angr.

\n

Reference: CTF Challenge Examples - angr documentation

\n

HackCon 2016 - angry-reverser

\n

Analyzing the binary in Ghidra reveals that it validates the input to determine whether the Flag is correct.

\n

However, the validation function performs extremely complex repeated computations as shown below, making it very difficult to identify the Flag through static analysis alone.

\n

\n \n \n \n \n \n \n \n \n

\n

When trying to find the correct input with angr, a simple SimulationManager script as shown in Symbolic Execution never finishes.

\n

So let’s read the example solve script:

\n
import angr\nimport claripy\n\ndef main():\n    flag    = claripy.BVS('flag', 20*8, explicit_name=True) # symbolized flag, we know the length by looking at the assembly code\n    buf     = 0x606000 # buffer to store flag\n    crazy   = 0x400646 # entry point of crazy function\n    find    = 0x405a6e # end of crazy function\n\n    # Offset of 'FAIL' blocks in Crazy(from pwntools--e.search(asm('mov ecx, 0')))\n    avoids = [0x402c3c, 0x402eaf, 0x40311c, 0x40338b, 0x4035f8, 0x403868,\n              0x403ad5, 0x403d47, 0x403fb9, 0x404227, 0x404496, 0x40470a,\n              0x404978, 0x404bec, 0x404e59, 0x4050c7, 0x405338, 0x4055a9,\n              0x4057f4, 0x405a2b]\n\n    proj = angr.Project('./yolomolo', auto_load_libs=False)\n    # Create blank state starting from crazy function\n    # LAZY_SOLVES is very important here because we are actually collecting constraints for an equation Ax=b, where A is 20 by 20, x and b are 20 by 1\n    state = proj.factory.blank_state(addr=crazy, add_options={angr.options.LAZY_SOLVES})\n    # insert flag into memory by hand\n    # state.memory.store(buf, flag, endness='Iend_LE')\n    state.memory.store(buf, flag, endness='Iend_BE')\n    state.regs.rdi = buf\n\n    # each character of flag should be between 0x30 and 0x7f\n    for i in range(19):\n        state.solver.add(flag.get_byte(i) >= 0x21)\n        state.solver.add(flag.get_byte(i) <= 0x7f)\n\n    simgr = proj.factory.simulation_manager(state)\n\n    simgr.explore(find=find, avoid=avoids)\n    found = simgr.found[0]\n    return found.solver.eval(flag, cast_to=bytes)\n\ndef test():\n    assert main() == b\"HACKCON{VVhYS04ngrY}\"\n\nif __name__ in '__main__':\n    import logging\n    logging.getLogger('angr.sim_manager').setLevel(logging.DEBUG)\n    print(main())
\n

This code creates a symbolic variable flag with a constraint that it consists of printable ASCII characters.

\n

It then stores the variable in big-endian format into a free area of the data section using state.memory.store(buf, flag, endness='Iend_BE'), sets that address in rdi, and creates a SimState with the Flag-validation function GoHomeOrGoCrazy as its entry point.

\n

This is a key technique worth remembering.

\n

SecurityFest 2016 - fairlight

\n

Next is a pattern where the Flag must be supplied as a command-line argument.

\n

Here, the Flag is identified by passing a symbolic variable as the args of entry_state.

\n
import angr\nimport claripy\n\ndef main():\n    proj = angr.Project('./fairlight', load_options={\"auto_load_libs\": False})\n    argv1 = claripy.BVS(\"argv1\", 0xE * 8)\n    state = proj.factory.entry_state(args=[\"./fairlight\", argv1]) \n\n    simgr = proj.factory.simulation_manager(state)\n    simgr.explore(find=0x4018f7, avoid=0x4018f9)\n    found = simgr.found[0]\n    return found.solver.eval(argv1, cast_to=bytes)\n\ndef test():\n    res = main()\n    print(repr(res))\n    assert res == b'4ngrman4gem3nt'\n\n\nif __name__ == '__main__':\n    import logging\n    logging.getLogger('angr.sim_manager').setLevel(logging.DEBUG)\n    print(main())
\n

Solving Constraints with z3py

\n

angr uses z3 internally as its SMT solver for constraint solving.

\n

Therefore, knowing how to write constraint-solving code in z3py is useful when using angr.

\n

You can also specify detailed constraints derived from decompilation results to identify the Flag.

\n

Constraint: Match a Popcount (Bit Sum) Function

\n
from z3 import *\nfrom math import *\n\ns = Solver()\ni1 = BitVec('i1', 64)\ni2 = BitVec('i2', 64)\n\n# Define a function to compute the sum of bits (popcount)\ndef HW(bvec):\n  return Sum([ZeroExt(int(ceil(log2(bvec.size()))), Extract(i,i,bvec)) for i in range(bvec.size())])\n\n# Define a function that returns 0 or 1 based on the popcount\ndef volcano(x):\n  w = HW(x)\n  return If(And(w > 16, w < 27), 1, 0)\n\ndef digitValue(c):\n    dv = StrToInt(c)\n    return If(dv == -1, 1, dv)\n\nif s.check() == sat:\n    print(s.model())\nelse:\n    print(\"unsat\")
\n

Constraint: Match the Number of Decimal Digits

\n
from z3 import *\n\ns = Solver()\ni1 = BitVec('i1', 64)\ni2 = BitVec('i2', 64)\n\n# Count decimal digits by converting to string\n# https://stackoverflow.com/questions/tagged/z3py\ni1 = BV2Int(i1)\ni2 = BV2Int(i2)\ns.add(Length(IntToStr(i1)) == Length(IntToStr(i2)))\n\nif s.check() == sat:\n    print(s.model())\nelse:\n    print(\"unsat\")
\n

Constraint: Match the Sum of Decimal Digits

\n
from z3 import *\nfrom math import *\n\ns = Solver()\ni1 = BitVec('i1', 64)\ni2 = BitVec('i2', 64)\n\n# Constraint that x and y have the same digit sum in decimal\n# https://stackoverflow.com/questions/76654949/z3-iterate-over-string-to-add-up-only-numbers-python-api\nadd_only_numbers = RecFunction('add_only_numbers', StringSort(), IntSort())\nw = FreshConst(StringSort())\nRecAddDefinition( add_only_numbers\n                , [w]\n                , If ( Length(w) == 0\n                     , 1\n                     , digitValue(Extract(w, 0, 1)) + add_only_numbers(Extract(w, 1, Length(w) - 1))\n                     )\n                )\ns.add(add_only_numbers(IntToStr(i1)) == add_only_numbers(IntToStr(i2)))\n\nif s.check() == sat:\n    print(s.model())\nelse:\n    print(\"unsat\")
\n

Shift and Rotate-Shift Operations in z3py

\n

You can perform shift operations on BitVec in z3py. (Regular Python shift operators also work.)

\n
# Create a 4-byte BitVec and add constraints on each character byte\nbuf = BitVec(\"buf\", 32)\nfor i in range(4):\n    # LShR(>>)\n    s.add((LShR(buf,8*i) & 0xFF) >= 0x21)\n    s.add((LShR(buf,8*i) & 0xFF) <= 0x7e)\n    \n# Similarly, to use only 3 bytes, specify constraints as follows\nbuf2 = BitVec(\"buf2\", 32)\ns.add(buf2 >> 24 == 0)\nfor i in range(3):\n    # LShR(>>)\n    s.add((LShR(buf2,8*i) & 0xFF) >= 0x30)\n    s.add((LShR(buf2,8*i) & 0xFF) <= 0x39)\n    \n    \n# Rotate-shift (equivalent to ror(1*b,15) in the following implementation)\n# def ror(a,b): return (LShR(a,b)|(a<<(32-b))) & N\nRotateRight((a * b), 15)\n\n# Equivalent to rol((a * b), 11) in the following implementation\n# def rol(a,b): return ror(a,32-b) # RotateLeft\nRotateLeft((a * b), 11)
\n

Reference: Z3: z3py Namespace Reference

\n

Solving CTF Problems with z3py

\n

Below are examples of solving CTF problem constraints with z3py.

\n

n00bz CTF 2023 - zzz

\n

Decompiling the given ELF binary in Ghidra produced the following result:

\n

\n \n \n \n \n \n \n \n \n

\n

I created a Solver with the following constraints based on the Ghidra decompilation and was able to obtain the Flag:

\n
from z3 import *\nflag = [BitVec(f\"flag[{i}]\", 8) for i in range(0x1e)]\ns = Solver()\n\n# Add custom constraints\ns.add(flag[0] == ord(\"n\"))\ns.add(flag[1] == ord(\"0\"))\ns.add(flag[2] == ord(\"0\"))\ns.add(flag[3] == ord(\"b\"))\ns.add(flag[4] == ord(\"z\"))\ns.add(flag[5] == ord(\"{\"))\ns.add(flag[0x1e-1] == ord(\"}\"))\nfor i in range(0x1e):\n    s.add(And(\n        (flag[i] >= 0x21),\n        (flag[i] <= 0x7e)\n    ))\n\n# Problem constraints\ns.add(flag[0] >> 4 == 0x6)\ns.add(flag[1] == flag[2])\ns.add(And(\n    ((flag[6] | flag[3]) == 0x7a),\n    ((flag[6] & flag[3]) == 0x42)\n))\ns.add(flag[0x1c] == flag[4]) \ns.add(And(\n    (flag[0x1d] * flag[5] == 0x3c0f),\n    And(\n        (flag[8] + flag[6] + flag[7] == 0x12e),\n        (flag[7] * flag[6] - flag[8] == 0x2a8a)\n    )\n))\n\ns.add(And(\n    And(\n        (flag[9] - flag[8] == 5),\n        (flag[10] - flag[9] == 0x1b),\n        (flag[10] ^ flag[0xb] == 0x20)\n    ),\n    And(\n        (flag[0xc] == flag[0xf]),\n        (flag[0xb] + flag[0xc] == 0xb4),\n        (flag[0xc] + flag[0xd] == 0xb9)\n    )\n))\n\ns.add(And(\n    (flag[0xd] + flag[0xe] - flag[0x10] == flag[0xd]),\n    And(\n        (flag[0x11] + flag[0x10] == 0xd9),\n        (flag[0x11] == flag[0xd])\n    ),\n    And(flag[0xe] + flag[0x10] == flag[0xe] * 2)\n))\n\ns.add(And(\n        And(\n            (flag[0x12] == ord('Z')),\n            (flag[0x12] == flag[0x13]),\n            ((flag[0x15] ^ flag[0x13] ^ flag[0x14]) == 0x7f)\n        ),\n        ((flag[0x14] ^ flag[0x15] ^ flag[0x16]) == flag[0x15]),\n        (flag[0x15] == ord('_')),\n        (flag[6] + flag[0x18] == 0xb4)\n))\n\ns.add(flag[0x18] + ~flag[0x17] == -0x21)\ns.add(flag[0x19] == flag[9])\ns.add(And(\n    (flag[0x1b] + flag[0x1a] == 0xd4),\n    (flag[0x1b] == flag[0x1c])\n))\n\nwhile s.check() == sat:\n    m = s.model()\n    for c in flag:\n        print(chr(m[c].as_long()),end=\"\")\n    print(\"\")\n    break
\n

SEKAI CTF - Guardians of the Kernel

\n\n

Reference: SEKAI CTF 2023 Writeup

\n

Summary

\n

I had been copy-pasting angr without understanding it, but in practice it turns out to be an extremely powerful tool — one that, including the content I couldn’t cover here, has the potential to be highly effective for solving all kinds of CTF problems.

\n

I plan to use this article as a cheat sheet and add to it as I go.

\n

Past Problems Solved with angr

\n\n

Past Problems Solved with z3

\n","fields":{"slug":"/ctf-angr-tutorial-en","tagSlugs":["/tag/ctf-en/","/tag/reversing-en/","/tag/angr-en/","/tag/z-3-en/","/tag/english/"]},"frontmatter":{"date":"2023-06-26","description":"I had been using angr with template scripts without much thought.","tags":["CTF (en)","Reversing (en)","angr (en)","Z3 (en)","English"],"title":"We Don't Know angr Yet (Z3py Notes Being Added)","socialImage":{"publicURL":"/static/eb679be4e5a7aef73e07814e275b0e99/ctf-angr-tutorial.png"}}}},"pageContext":{"slug":"/ctf-angr-tutorial-en"}},"staticQueryHashes":["251939775","401334301","825871152"]}