--- class: center, middle # Testcase generation --- # Testcase generation http://doc.ntp.org/4.1.0/ntpq.htm --- # Steal --- ``` dude@dudebox:~/projects/ntpd/run/in$ ls -l | wc -l 43 dude@dudebox:~/projects/ntpd/run/in$ ls -l total 168 -rw-r--r-- 2 dude dude 68 Jun 18 11:57 decodenetnumtrigger1.raw -rw-r--r-- 2 dude dude 12 Jun 18 11:57 ntpassociations -rw-r--r-- 1 dude dude 36 Jun 18 11:57 ntpauth -rw-r--r-- 4 dude dude 12 Jun 18 11:57 ntpbeginningstrange -rw-r--r-- 2 dude dude 20 Jun 18 11:57 ntpclockvarassocid -rw-r--r-- 2 dude dude 24 Jun 18 11:57 ntpclockvarassocidset -rw-r--r-- 2 dude dude 24 Jun 18 11:57 ntpclockvarbadformat -rw-r--r-- 2 dude dude 24 Jun 18 11:57 ntpclockvarbadformatset -rw-r--r-- 2 dude dude 20 Jun 18 11:57 ntpclockvardevice -rw-r--r-- 2 dude dude 24 Jun 18 11:57 ntpclockvardeviceclock -rw-r--r-- 2 dude dude 24 Jun 18 11:57 ntpclockvardevicelocal -rw-r--r-- 2 dude dude 32 Jun 18 11:57 ntpclockvardeviceundisciplined -rw-r--r-- 2 dude dude 20 Jun 18 11:57 ntpclockvarflags -rw-r--r-- 7 dude dude 20 Jun 18 11:57 ntpclockvarflagsset -rw-r--r-- 8 dude dude 16 Jun 18 11:57 ntpclockvarpoll -rw-r--r-- 4 dude dude 20 Jun 18 11:57 ntpclockvarpollset -rw-r--r-- 7 dude dude 20 Jun 18 11:57 ntpclockvarstatus -rw-r--r-- 7 dude dude 20 Jun 18 11:57 ntpclockvartimecode -rw-r--r-- 2 dude dude 24 Jun 18 11:57 ntpclockvartimecodeset -rw-r--r-- 7 dude dude 104 Jun 18 11:57 ntpmonstats -rw-r--r-- 2 dude dude 52 Jun 18 11:57 ntpmrulist -rw-r--r-- 2 dude dude 68 Jun 18 11:57 ntpmrulistkod -rw-r--r-- 2 dude dude 72 Jun 18 11:57 ntpmrulistladdrset -rw-r--r-- 2 dude dude 68 Jun 18 11:57 ntpmrulistlimited -rw-r--r-- 2 dude dude 52 Jun 18 11:57 ntpmrulistmincount -rw-r--r-- 2 dude dude 68 Jun 18 11:57 ntpmrulistmincountset -rw-r--r-- 2 dude dude 68 Jun 18 11:57 ntpmrulistresallhexmask -rw-r--r-- 4 dude dude 68 Jun 18 11:57 ntpmrulistresanyhexmask -rw-r--r-- 2 dude dude 52 Jun 18 11:57 ntpmrulistsortorderaddr -rw-r--r-- 2 dude dude 52 Jun 18 11:57 ntpmrulistsortorderavgint -rw-r--r-- 2 dude dude 52 Jun 18 11:57 ntpmrulistsortordercount -rw-r--r-- 2 dude dude 52 Jun 18 11:57 ntpmrulistsortorderlstint -rw-r--r-- 2 dude dude 52 Jun 18 11:57 ntpmrulistsortorderlstintreverse -rw-r--r-- 4 dude dude 12 Jun 18 11:57 ntppeers -rw-r--r-- 4 dude dude 12 Jun 18 11:57 ntppeerschallengeresponse -rw-r--r-- 2 dude dude 16 Jun 18 11:57 ntpreadvarpeer -rw-r--r-- 2 dude dude 24 Jun 18 11:57 ntpreadvarprocessor -rw-r--r-- 4 dude dude 20 Jun 18 11:57 ntpreadvarstatus -rw-r--r-- 7 dude dude 20 Jun 18 11:57 ntpreadvaversion -rw-r--r-- 1 dude dude 44 Jun 18 11:57 ntpwritevarpeer -rw-r--r-- 1 dude dude 52 Jun 18 11:57 ntpwritevarrootdisp -rw-r--r-- 2 dude dude 48 Jun 18 11:57 raw ``` --- class: center, middle # Measuring coverage "What code did I actually fuzz?" --- # afl-cov - https://github.com/mrash/afl-cov - https://www.cipherdyne.org/fwknop/2.6.7-afl-lcov-results/ - https://www.cipherdyne.org/fwknop/2.6.7-afl-lcov-results/server/replay_cache.c.gcov.html Sucks to figure out that you have been fuzzing the checksum check for multiple weeks Example: PNG --- class: center, middle # (more) Error detection --- # (more) Error detection - libdislocator - ASan - Address Sanitizer - libdislocator++ harder to use + insignificant slowdown - valgrind - ASan++, but HUGE slowdown - MSan - Memory Santizer - TSan - Thread Sanitizer - UBSan - Undefined Behavior Sanitizer - KASan - Kernel Address Sanitizer --- # (more) Error detection - **libdislocator** - **ASan** - Address Sanitizer - libdislocator++ harder to use + insignificant slowdown - valgrind - ASan++, but HUGE slowdown - MSan - Memory Santizer - TSan - Thread Sanitizer - UBSan - Undefined Behavior Sanitizer - KASan - Kernel Address Sanitizer --- # libdislocator Usage: AFL_LD_PRELOAD=/path/to/libdislocator.so ./afl-fuzz [...other params...] https://github.com/mcarpenter/afl/tree/master/libdislocator ``` 1) It allocates all buffers so that they are immediately adjacent to a subsequent PROT_NONE page, causing most off-by-one reads and writes to immediately segfault, 2) It adds a canary immediately below the allocated buffer, to catch writes to negative offsets (won't catch reads, though), 3) It sets the memory returned by malloc() to garbage values, improving the odds of crashing when the target accesses uninitialized data, 4) It sets freed memory to PROT_NONE and does not actually reuse it, causing most use-after-free bugs to segfault right away, 5) It forces all realloc() calls to return a new address - and sets PROT_NONE on the original block. This catches use-after-realloc bugs, 6) It checks for calloc() overflows and can cause soft or hard failures of alloc requests past a configurable memory limit (AFL_LD_LIMIT_MB, AFL_LD_HARD_LIMIT). Basically, it is inspired by some of the non-default options available for the OpenBSD allocator. It is meant as a more lightweight and hassle-free alternative to fuzzing with ASAN / MSAN (although it's obviously not as comprehensive). ``` --- class: center, middle # Address Sanitizer (ASan) = libdislocator + read checks + **more** ``` # AFL_USE_ASAN=1 ... ``` --- # ASan + 64-bit != 3 --- # docs/notes_for_asan.txt ``` 2) Long version --------------- ASAN allocates a huge region of virtual address space for bookkeeping purposes. Most of this is never actually accessed, so the OS never has to allocate any real pages of memory for the process, and the VM grabbed by ASAN is essentially "free" - but the mapping counts against the standard OS-enforced limit (RLIMIT_AS, aka ulimit -v). On our end, afl-fuzz tries to protect you from processes that go off-rails and start consuming all the available memory in a vain attempt to parse a malformed input file. This happens surprisingly often, so enforcing such a limit is important for almost any fuzzer: the alternative is for the kernel OOM handler to step in and start killing random processes to free up resources. Needless to say, that's not a very nice prospect to live with. ``` "On 64-bit systems, the situation is more murky, because the ASAN allocation is completely outlandish - around 17.5 TB in older versions, and closer to **20 TB** with newest ones." --- --- # afl/experimental/asan_cgroups/limit_memory.sh 1. enable cgroups in kernel (grub boot options) 2. `# swapoff -a` ``` # sudo ./limit_memory.sh -u dude -m 500 -- ./afl-fuzz.. ``` --- --- # ASan --- # Valgrind --- class: center, middle # Compiler transformation for more code coverage https://lafintel.wordpress.com/2016/08/15/circumventing-fuzzing-roadblocks-with-compiler-transformations/ --- class: center, middle # Dictionaries https://github.com/mcarpenter/afl/tree/master/dictionaries --- class: center, middle # Test case minification - Fuzzer maintenance --- # Test case minification afl-cmin: https://github.com/mirrorer/afl/blob/master/afl-cmin ``` # This tool tries to find the smallest subset of files in the input directory # that still trigger the full range of instrumentation data points seen in # the starting corpus. This has two uses: # # - Screening large corpora of input files before using them as a seed for # afl-fuzz. The tool will remove functionally redundant files and likely # leave you with a much smaller set. # # (In this case, you probably also want to consider running afl-tmin on # the individual files later on to reduce their size.) # # - Minimizing the corpus generated organically by afl-fuzz, perhaps when # planning to feed it to more resource-intensive tools. The tool achieves # this by removing all entries that used to trigger unique behaviors in the # past, but have been made obsolete by later finds. ``` --- # Test case minification afl-tmin https://github.com/mirrorer/afl/blob/master/afl-tmin.c ``` A simple test case minimizer that takes an input file and tries to remove as much data as possible while keeping the binary in a crashing state *or* producing consistent instrumentation output (the mode is auto-selected based on the initially observed behavior). ``` --- class: center, middle # Corpus driven fuzzing term coined by Ben Nagy (@rantyben)? https://github.com/bnagy/slides/blob/master/fuzzing_without_pub.pdf --- # Corpus driven fuzzing - ms15-024 / ms15-029 found by lcamtuf found in IE without ever fuzzing IE - Goal: build good corpora. NOT finding crashes e.g. 1. fuzz pdf parser A 2. take found corpora/testcases and fuzz pdf parser B 3. profit --- class: center, middle # Beyond crashes A fuzzer is good at finding crashes, so let's convert whatever we want to find into a crash. --- # Beyond crashes ``` ABORT(3) Linux Programmer's Manual ABORT(3) NAME abort - cause abnormal process termination SYNOPSIS #include void abort(void); DESCRIPTION The abort() first unblocks the SIGABRT signal, and then raises that signal for the calling process. This results in the abnormal termination of the process unless the SIGABRT signal is caught and the signal handler does not return (see longjmp(3)). If the abort() function causes process termination, all open streams are closed and flushed. If the SIGABRT signal is ignored, or caught by a handler that returns, the abort() function will still terminate the process. It does this by restoring the default disposition for SIGABRT and then raising the signal for a second time. ``` --- # Logic flaws ``` input = readInput(); resultA = BigNumLibraryA_call(input); resultB = BigNumLibraryB_call(input); if (resultA != resultB) { abort(); } exit(0); ``` --- # Authentication bypass ``` ... if (authenticated == true) { abort(); } exit(0); ``` --- # Be creative! - sandbox escape? - side channels? - degradation of service? - mess up audit trail? - escalation of privileges? - +++ - insert crazy idea here If it can be converted to a crash, then it can be fuzzed --- class: center, middle # Targeting --- # Targeting - do something different - .. and be the first to do it e.g. be creative and don't wait --- # Targeting - $$$ - curiosity - both?? aka "why do you spend your evenings on infosec?" --- --- # references - slides: http://dumpco.re/afl - http://imgur.com/a/O7z5F - strategies: https://lcamtuf.blogspot.dk/2014/08/binary-fuzzing-strategies-what-works.html - fuzz upx https://asciinema.org/a/e7bpjng8jj33o53qmctkihka8 - fuzz ntpd https://asciinema.org/a/1npswngnfah6m4m0et246e0lr - auxiliary tools by Ben Nagy (@rantyben) https://github.com/bnagy?tab=repositories