+ - 0:00:00
Notes for current slide
Notes for next slide

AFL

American Fuzzy Lop

slides: http://dumpco.re/afl

1 / 81

2 / 81

Agenda

  1. Why care?
  2. Why it's better
  3. "hello world" setup
  4. Why it's faster
  5. How to fuzz a server
  6. Testcase generation
  7. measuring coverage with afl-cov
  8. Error detection
  9. Compiler transformation for better coverage
  10. Dictionaries
  11. Test case minification aka fuzzer maintenance
  12. "Corpus driven fuzzing"
  13. Beyond crashes
  14. Targeting
3 / 81

Why it's better

5 / 81

6 / 81

7 / 81

"Hello world" setup

9 / 81

"Hello world" setup

# CC=~/tools/afl-2.35b/afl-clang-fast \
CXX=~/tools/afl-2.35b/afl-clang-fast++ make clean all
# ~/tools/afl-2.35b/afl-fuzz -i testcases/ \
-o output -- ./upx-3.91-src/src/upx.out -d @@
10 / 81

Demo

11 / 81
12 / 81

dude@dudebox:~/projects/demo$ tree output/

output/
├── crashes
│ ├── id:000000,sig:06,src:000000,op:flip1,pos:201
│ ├── id:000001,sig:06,src:000000,op:flip1,pos:205
│ ├── id:000002,sig:06,src:000000,op:flip1,pos:206
│ ├── id:000003,sig:06,src:000000,op:flip1,pos:206
│ ├── id:000004,sig:06,src:000000,op:flip1,pos:206
│ └── README.txt
├── fuzz_bitmap
├── fuzzer_stats
├── hangs
├── plot_data
└── queue
├── id:000000,orig:ls.compressed
├── id:000001,src:000000,op:flip1,pos:0,+cov
├── id:000002,src:000000,op:flip1,pos:4
├── id:000003,src:000000,op:flip1,pos:5,+cov
├── id:000004,src:000000,op:flip1,pos:6,+cov
...
├── id:000089,src:000000,op:flip1,pos:4101
├── id:000090,src:000000,op:flip1,pos:4384
└── id:000091,src:000000,op:flip1,pos:4764
3 directories, 101 files
13 / 81

Performance optimisations

14 / 81

Performance optimisations

  1. Fork server
  2. Deferred initialisation
  3. Persistent mode
15 / 81

traditional fuzzing with execve() - How executables normally are started

  1. read executable file from disk
  2. parse executable file
  3. init virtual memory
  4. init stack
  5. init heap
  6. load shared libraries (.dll .so .dylib)
  7. +++
  8. call main()
16 / 81

traditional fuzzing with execve() - How executables normally are started

  1. read executable file from disk <- couldn't care less
  2. parse executable file <- couldn't care less
  3. init virtual memory <- couldn't care less
  4. init stack <- couldn't care less
  5. init heap <- couldn't care less
  6. load shared libraries (.dll .so .dylib) <- couldn't care less
  7. +++ <- couldn't care less
  8. call main()

'

17 / 81

18 / 81

Fork server

19 / 81

Fork server

  1. read executable file from disk
  2. parse executable file
  3. init virtual memory
  4. init stack
  5. init heap
  6. load shared libraries (.dll .so .dylib)
  7. +++
  8. call main()
20 / 81

Fork server

  1. read executable file from disk
  2. parse executable file
  3. init virtual memory
  4. init stack
  5. init heap
  6. load shared libraries (.dll .so .dylib)
  7. +++
  8. fork()
  9. call main()
21 / 81

Fork server

  1. fork()
  2. main()
22 / 81

works automatically!

23 / 81

But, we want more

24 / 81

Fork server

  1. fork()
  2. main()
25 / 81

Fork server

  1. fork()
  2. main()
    • parse cli args
    • readConfig()
    • initStuff()
    • check for updates
    • calculate more stuff
    • +++
    • readInput()
    • parseInput()
26 / 81

Deferred initialisation

  1. fork()
  2. main()
    • parse cli args
    • readConfig()
    • initStuff()
    • check for updates
    • calculate more stuff
    • +++
    • fork()
    • readInput()
    • parseInput()
27 / 81

Deferred initialisation

  1. fork()
  2. main()
    • parse cli args
    • readConfig()
    • initStuff()
    • check for updates
    • calculate more stuff
    • +++
    • fork()
    • readInput()
    • parseInput()
      #ifdef __AFL_HAVE_MANUAL_CONTROL
      __AFL_INIT();
      #endif
      readInput()
      parseInput()
28 / 81

But, we want MORE

29 / 81

Persistent mode

30 / 81

Deferred initialisation

  1. fork()
  2. readInput()
  3. parseInput()
31 / 81

Persistent mode

  1. fork()
  2. readInput()
  3. parseInput()
32 / 81

Persistent mode

  1. fork()
  2. readInput()
  3. parseInput()
init()
while (__AFL_LOOP(1000)) {
readInput();
parseInput();
}
exit(0);
33 / 81

Server fuzzing

34 / 81

Server fuzzing

init();
while (keep_running) {
waitForData(); // blocking
readInput();
parseInput();
housekeeping();
}
cleanup();
35 / 81

Server fuzzing

init();
//while (keep_running) {
//waitForData(); // blocking
//readInput();
readFromStdin(); (or) readFromFile();
parseInput();
exit(0);
housekeeping();
//}
cleanup();
36 / 81

Server fuzzing + deferred initialisation

init();
//while (keep_running) {
//waitForData(); // blocking
//readInput();
#ifdef __AFL_HAVE_MANUAL_CONTROL
__AFL_INIT();
#endif
readFromStdin(); (or) readFromFile();
parseInput();
exit(0);
housekeeping();
//}
cleanup();
37 / 81

Server fuzzing + persistent mode

init();
//while (keep_running) {
while (__AFL_LOOP(1000) {
//waitForData(); // blocking
//readInput();
readFromStdin(); (or) readFromFile();
parseInput();
housekeeping();
}
exit(0);
cleanup();
38 / 81

39 / 81

Lazy server fuzzing + persistent mode

init();
//while (keep_running) {
while (__AFL_LOOP(1000) {
input = readFromStdin(); (or) readFromFile();
sendToItself(input);
waitForData(); // blocking
readInput();
parseInput();
housekeeping();
}
exit(0);
cleanup();
40 / 81

Fuzzing ntpd

network time protocol daemon

41 / 81

ntpd/ntpd.c

for (;;) {
#if !defined(SIM) && defined(SIGDIE1)
if (signalled)
finish_safe(signo);
#endif
if (alarm_flag) { /* alarmed? */
was_alarmed = TRUE;
alarm_flag = FALSE;
}
/* collect async name/addr results */
if (!was_alarmed)
harvest_blocking_responses();
if (!was_alarmed && !has_full_recv_buffer()) {
/*
* Nothing to do. Wait for something.
*/
io_handler();
}
42 / 81

ntpd/ntpd.c - modified

#define BUFLEN 5120
struct sockaddr_in si_other;
int s, slen=sizeof(si_other);
char buf[BUFLEN];
if ((s=socket(AF_INET, SOCK_DGRAM, IPPROTO_UDP))==-1) {
perror("socket");
abort();
}
memset((char *) &si_other, 0, sizeof(si_other));
si_other.sin_family = AF_INET;
si_other.sin_addr.s_addr = inet_addr("127.0.0.1");
si_other.sin_port = htons(123);
while (__AFL_LOOP(1000)) {
...
if (!was_alarmed && !has_full_recv_buffer()) {
memset(buf, 0, BUFLEN);
size_t insize = read(0, buf, BUFLEN);
if (sendto(s, buf, insize, 0, (struct sockaddr *)&si_other, slen)==-1) {
perror("sendto()");
abort();
}
io_handler();
}
43 / 81

Demo

44 / 81
45 / 81

afl-plot

46 / 81

Testcase generation

47 / 81

Testcase generation

http://doc.ntp.org/4.1.0/ntpq.htm

48 / 81

Steal

49 / 81
dude@dudebox:~/projects/ntpd/run/in$ ls -l | wc -l
43
dude@dudebox:~/projects/ntpd/run/in$ ls -l
total 168
-rw-r--r-- 2 dude dude 68 Jun 18 11:57 decodenetnumtrigger1.raw
-rw-r--r-- 2 dude dude 12 Jun 18 11:57 ntpassociations
-rw-r--r-- 1 dude dude 36 Jun 18 11:57 ntpauth
-rw-r--r-- 4 dude dude 12 Jun 18 11:57 ntpbeginningstrange
-rw-r--r-- 2 dude dude 20 Jun 18 11:57 ntpclockvarassocid
-rw-r--r-- 2 dude dude 24 Jun 18 11:57 ntpclockvarassocidset
-rw-r--r-- 2 dude dude 24 Jun 18 11:57 ntpclockvarbadformat
-rw-r--r-- 2 dude dude 24 Jun 18 11:57 ntpclockvarbadformatset
-rw-r--r-- 2 dude dude 20 Jun 18 11:57 ntpclockvardevice
-rw-r--r-- 2 dude dude 24 Jun 18 11:57 ntpclockvardeviceclock
-rw-r--r-- 2 dude dude 24 Jun 18 11:57 ntpclockvardevicelocal
-rw-r--r-- 2 dude dude 32 Jun 18 11:57 ntpclockvardeviceundisciplined
-rw-r--r-- 2 dude dude 20 Jun 18 11:57 ntpclockvarflags
-rw-r--r-- 7 dude dude 20 Jun 18 11:57 ntpclockvarflagsset
-rw-r--r-- 8 dude dude 16 Jun 18 11:57 ntpclockvarpoll
-rw-r--r-- 4 dude dude 20 Jun 18 11:57 ntpclockvarpollset
-rw-r--r-- 7 dude dude 20 Jun 18 11:57 ntpclockvarstatus
-rw-r--r-- 7 dude dude 20 Jun 18 11:57 ntpclockvartimecode
-rw-r--r-- 2 dude dude 24 Jun 18 11:57 ntpclockvartimecodeset
-rw-r--r-- 7 dude dude 104 Jun 18 11:57 ntpmonstats
-rw-r--r-- 2 dude dude 52 Jun 18 11:57 ntpmrulist
-rw-r--r-- 2 dude dude 68 Jun 18 11:57 ntpmrulistkod
-rw-r--r-- 2 dude dude 72 Jun 18 11:57 ntpmrulistladdrset
-rw-r--r-- 2 dude dude 68 Jun 18 11:57 ntpmrulistlimited
-rw-r--r-- 2 dude dude 52 Jun 18 11:57 ntpmrulistmincount
-rw-r--r-- 2 dude dude 68 Jun 18 11:57 ntpmrulistmincountset
-rw-r--r-- 2 dude dude 68 Jun 18 11:57 ntpmrulistresallhexmask
-rw-r--r-- 4 dude dude 68 Jun 18 11:57 ntpmrulistresanyhexmask
-rw-r--r-- 2 dude dude 52 Jun 18 11:57 ntpmrulistsortorderaddr
-rw-r--r-- 2 dude dude 52 Jun 18 11:57 ntpmrulistsortorderavgint
-rw-r--r-- 2 dude dude 52 Jun 18 11:57 ntpmrulistsortordercount
-rw-r--r-- 2 dude dude 52 Jun 18 11:57 ntpmrulistsortorderlstint
-rw-r--r-- 2 dude dude 52 Jun 18 11:57 ntpmrulistsortorderlstintreverse
-rw-r--r-- 4 dude dude 12 Jun 18 11:57 ntppeers
-rw-r--r-- 4 dude dude 12 Jun 18 11:57 ntppeerschallengeresponse
-rw-r--r-- 2 dude dude 16 Jun 18 11:57 ntpreadvarpeer
-rw-r--r-- 2 dude dude 24 Jun 18 11:57 ntpreadvarprocessor
-rw-r--r-- 4 dude dude 20 Jun 18 11:57 ntpreadvarstatus
-rw-r--r-- 7 dude dude 20 Jun 18 11:57 ntpreadvaversion
-rw-r--r-- 1 dude dude 44 Jun 18 11:57 ntpwritevarpeer
-rw-r--r-- 1 dude dude 52 Jun 18 11:57 ntpwritevarrootdisp
-rw-r--r-- 2 dude dude 48 Jun 18 11:57 raw
50 / 81

Measuring coverage

"What code did I actually fuzz?"

51 / 81

(more) Error detection

53 / 81

(more) Error detection

  • libdislocator
  • ASan - Address Sanitizer - libdislocator++ harder to use + insignificant slowdown
  • valgrind - ASan++, but HUGE slowdown
  • MSan - Memory Santizer
  • TSan - Thread Sanitizer
  • UBSan - Undefined Behavior Sanitizer
  • KASan - Kernel Address Sanitizer
54 / 81

(more) Error detection

  • libdislocator
  • ASan - Address Sanitizer - libdislocator++ harder to use + insignificant slowdown
  • valgrind - ASan++, but HUGE slowdown
  • MSan - Memory Santizer
  • TSan - Thread Sanitizer
  • UBSan - Undefined Behavior Sanitizer
  • KASan - Kernel Address Sanitizer
55 / 81

libdislocator

Usage: AFL_LD_PRELOAD=/path/to/libdislocator.so ./afl-fuzz [...other params...] https://github.com/mcarpenter/afl/tree/master/libdislocator

1) It allocates all buffers so that they are immediately adjacent to a
subsequent PROT_NONE page, causing most off-by-one reads and writes to
immediately segfault,
2) It adds a canary immediately below the allocated buffer, to catch
writes to negative offsets (won't catch reads, though),
3) It sets the memory returned by malloc() to garbage values,
improving the odds of crashing when the target accesses uninitialized
data,
4) It sets freed memory to PROT_NONE and does not actually reuse it,
causing most use-after-free bugs to segfault right away,
5) It forces all realloc() calls to return a new address - and sets
PROT_NONE on the original block. This catches use-after-realloc bugs,
6) It checks for calloc() overflows and can cause soft or hard
failures of alloc requests past a configurable memory limit
(AFL_LD_LIMIT_MB, AFL_LD_HARD_LIMIT).
Basically, it is inspired by some of the non-default options available
for the OpenBSD allocator. It is meant as a more lightweight and
hassle-free alternative to fuzzing with ASAN / MSAN (although it's
obviously not as comprehensive).
56 / 81

Address Sanitizer (ASan)

= libdislocator + read checks + more

# AFL_USE_ASAN=1 ...
57 / 81

ASan + 64-bit != </3

58 / 81

docs/notes_for_asan.txt

2) Long version
---------------
ASAN allocates a huge region of virtual address space for bookkeeping purposes.
Most of this is never actually accessed, so the OS never has to allocate any
real pages of memory for the process, and the VM grabbed by ASAN is essentially
"free" - but the mapping counts against the standard OS-enforced limit
(RLIMIT_AS, aka ulimit -v).
On our end, afl-fuzz tries to protect you from processes that go off-rails
and start consuming all the available memory in a vain attempt to parse a
malformed input file. This happens surprisingly often, so enforcing such a limit
is important for almost any fuzzer: the alternative is for the kernel OOM
handler to step in and start killing random processes to free up resources.
Needless to say, that's not a very nice prospect to live with.

"On 64-bit systems, the situation is more murky, because the ASAN allocation is completely outlandish - around 17.5 TB in older versions, and closer to 20 TB with newest ones."

59 / 81

60 / 81

afl/experimental/asan_cgroups/limit_memory.sh

  1. enable cgroups in kernel (grub boot options)
  2. # swapoff -a
# sudo ./limit_memory.sh -u dude -m 500 -- ./afl-fuzz..
61 / 81
62 / 81

ASan

63 / 81

Valgrind

64 / 81

Test case minification - Fuzzer maintenance

67 / 81

Test case minification

afl-cmin: https://github.com/mirrorer/afl/blob/master/afl-cmin

# This tool tries to find the smallest subset of files in the input directory
# that still trigger the full range of instrumentation data points seen in
# the starting corpus. This has two uses:
#
# - Screening large corpora of input files before using them as a seed for
# afl-fuzz. The tool will remove functionally redundant files and likely
# leave you with a much smaller set.
#
# (In this case, you probably also want to consider running afl-tmin on
# the individual files later on to reduce their size.)
#
# - Minimizing the corpus generated organically by afl-fuzz, perhaps when
# planning to feed it to more resource-intensive tools. The tool achieves
# this by removing all entries that used to trigger unique behaviors in the
# past, but have been made obsolete by later finds.
68 / 81

Test case minification

afl-tmin https://github.com/mirrorer/afl/blob/master/afl-tmin.c

A simple test case minimizer that takes an input file and tries to remove
as much data as possible while keeping the binary in a crashing state
*or* producing consistent instrumentation output (the mode is auto-selected
based on the initially observed behavior).
69 / 81

Corpus driven fuzzing

term coined by Ben Nagy (@rantyben)? https://github.com/bnagy/slides/blob/master/fuzzing_without_pub.pdf

70 / 81

Corpus driven fuzzing

  • ms15-024 / ms15-029 found by lcamtuf found in IE without ever fuzzing IE
  • Goal: build good corpora. NOT finding crashes

e.g.

  1. fuzz pdf parser A
  2. take found corpora/testcases and fuzz pdf parser B
  3. profit
71 / 81

Beyond crashes

A fuzzer is good at finding crashes, so let's convert whatever we want to find into a crash.

72 / 81

Beyond crashes

ABORT(3) Linux Programmer's Manual ABORT(3)
NAME
abort - cause abnormal process termination
SYNOPSIS
#include <stdlib.h>
void abort(void);
DESCRIPTION
The abort() first unblocks the SIGABRT signal, and then raises that signal
for the calling process. This results in the abnormal termination of the
process unless the SIGABRT signal is caught and the signal handler does not
return (see longjmp(3)).
If the abort() function causes process termination, all open streams are
closed and flushed.
If the SIGABRT signal is ignored, or caught by a handler that returns, the
abort() function will still terminate the process. It does this by restoring
the default disposition for SIGABRT and then raising the signal for a second
time.
73 / 81

Logic flaws

input = readInput();
resultA = BigNumLibraryA_call(input);
resultB = BigNumLibraryB_call(input);
if (resultA != resultB) {
abort();
}
exit(0);
74 / 81

Authentication bypass

...
if (authenticated == true) {
abort();
}
exit(0);
75 / 81

Be creative!

  • sandbox escape?
  • side channels?
  • degradation of service?
  • mess up audit trail?
  • escalation of privileges?
  • +++
  • insert crazy idea here

If it can be converted to a crash, then it can be fuzzed

76 / 81

Targeting

77 / 81

Targeting

  • do something different
  • .. and be the first to do it

e.g. be creative and don't wait

78 / 81

Targeting

  • $$$
  • curiosity
  • both??

aka "why do you spend your evenings on infosec?"

79 / 81

80 / 81

2 / 81
Paused

Help

Keyboard shortcuts

, , Pg Up, k Go to previous slide
, , Pg Dn, Space, j Go to next slide
Home Go to first slide
End Go to last slide
Number + Return Go to specific slide
b / m / f Toggle blackout / mirrored / fullscreen mode
c Clone slideshow
p Toggle presenter mode
t Restart the presentation timer
?, h Toggle this help
Esc Back to slideshow