CSL865: Spl. Topics in Computer Applications: Systems Security

Sem II, 2009-10

Programming Assignments

1. Buffer Overflow Exploits
2. Fuzzing (due date: March 19, 2010)
3. Traceroute (due date: April 17, 2010)

Readings

Overview

• Reflections on Trusting Trust, Ken Thompson
• Botnets, Cybercrime, and Cyberterrorism: Vulnerabilities and Policy Issues for Congress (Congressional Research Services Report)

Secure system design

• Shifting the Odds: Writing (More) Secure Software, slides by S. Bellovin, 1996
• Optional: Bypassing Browser Memory Protections, A. Sotirov (a detailed analysis of Windows memory protection)

Buffer Overflows, Integer Overflows, formatstring vulnerabilties, other libc bugs [PDF, PPT]

• Smashing the stack for Fun and Profit, Aleph One
• The Confused Deputy, Norm Hardy
• Basic Integer Overflows, blexim
• Format string attack, wikipedia.
• mktemp() TOCTOU bug

Access Control and Protection [PDF, PPT]

• Role-based Access Control, R. Sandhu et al., IEEE Computer 1996
• Proposed NIST Standard for RBAC. ACM TISSEC, 4(3):224-274, Aug. 2001.
• Kerberos description on wikipedia

Defenses against well-known attacks (buffer overflows, etc.) [PDF, PPT]

• StackGuard: Automatic Adaptive Detection and Prevention of Buffer-Overflow Attacks, Cowan et al., 1998
• ASLR Smack and Laugh Reference, T. Muller 2008
• Data Execution Protection (DEP)/SafeSEH, wikipedia.
• PointGuard: Protecting Pointers from Buffer Overflow Vulnerabilities, C. Cowan et al. Usenix Security 2003

Heap Spraying / Stack Spraying Attacks [PDF, PPT]

• Heap Feng Shui in Javascript, A. Sotriov 2007
• Engineering Heap Overflow Exploits with JavaScript, M. Daniel, J. Honoroff. WooT 2008
• Nozzle: A Defense Against Heap-Spraying Code Injection Attacks, P. Ratanaworabhan et al. (MSR). November 2008
• Bypassic Browser Memory Protections, A. Sotirov, M. Dowd. 2008

Testing for security via fuzzing

• Real world fuzzing, by Charlie Miller
• Effective Bug Discovery, vf.
• How hackers look for bugs, by Dave Aitel

Tools for writing robust application code

• KLEE: Unassisted and Automatic Generation of High-Coverage Tests for Complex Systems Programs, by C. Cadar, D. Dunbar, D. Engler.
• Finding Security Vulnerabilities in Java Applications with Static Analysis, by B. Livshits and M. Lam.
• Using Programmer-Written Compiler Extensions to Catch Security Holes, by K. Ashcraft, D. Engler.
• Proof-Carrying Code, G. Necula, 1997

Dealing with bad (legacy) application code: sandboxing and isolation [PDF, PPT]

• Traps and Pitfalls: Practical Problems in System Call Interposition Based Security Tools, T. Garfinkel
• Efficient Software-Based Fault Isolation, Robert Wahbe, et al.
• A note on the confinement problem, Butler Lampson

Virtualization

• A Virtual Machine Introspection Based Architecture for Intrusion Detection, T. Garfinkel, M. Rosenblum. NDSS 2003
• SubVirt: Implementing Malware with Virtual Machines, S. King, P. Chen et al. 2006
• Compatibility is Not Transparency: VMM Detection Myths and Realities, T. Garfkinkel et al. HotOS 2007
• Overshadow: A Virtualization-Based Approach to Retrofitting Protection in Commodity Operating Systems. X. Chen et al. 2008
• Secure Virtual Architecture: A Safe Execution Environment for Commodity Operating Systems, J. Criswell et al. 2007

Data Lifetime

• Understanding Data Lifetime via Whole System Simulation, J. Chow et al. 2004.

OS abstractions for Browsers

• Protection and Communication Abstractions for Web Browsers in Mashup OS, Wang et al. (MSR), 2007
• A Safety-Oriented Platform for Web Applications, Cox et al. (U. Washington), 2006

Use of cryptography in computer security [PDF, PPT]

• Why cryptosystems fail, Ross Anderson

Security problems in network protocols: TCP, DNS, SMTP, and routing [PDF, PPT]

• A look back at security problems in TCP/IP Protocol Suite, S. Bellovin, ACSAC 2004
• A simple active attack against TCP, Joncheray, 1995
• A survey of BGP security, 2005
• DNS cache poisoning, Steve Friedl
• Improving the security of your site by breaking into it, Farmer and Venema, 1995
• Using the Domain Name System for System Break-ins, Bellovin, 1995.

Network defense tools: Firewalls, VPNs, Intrusion Detection, and filters [PDF, PPT]

• DNSSEC - The Theory, G. Huston
• Network Firewalls, S.M. Bellovin and W.R. Cheswick
• Bro: A system for detecting network intruders in real-time, V. Paxon

Malware: Computer viruses, spyware, and key-loggers [PDF, PPT]

• Hunting for metamorphic, Szor, P. Ferrie

Bot-nets: Attacks and Defenses [PDF, PPT]

• Inside the slammer worm, S. Savage
• An Analysis of Conficker's Logic and Rendezvous Points
• Know Your Enemy: Fast-Flux Service Networks, Honeynet

Unwanted Traffic: Denial-of-Service attacks and Spam email [PDF, PPT]

• Practical network support for IP Traceback, S. Savage, et al.
• A DoS-Limiting Network Architecture, Yang, Wetherall, and Anderson
• A detailed DDoS extortion story

Network security testing [PDF, PPT]

• The Art of Port Scanning, Fyodor
• pOf_2: Dr. Jekyll had something to Hyde, Zalewski

Mobile Networks

• GSM hack--operator flunks the challenge, Anderson, 1997
• GSM Interception, Pesonen, 1999

Anonymization

• Solutions for Anonymous Communication on the Internet, Claessens, Preneel, Vandewalle, 1999

Basic web security model [PDF, PPT]

• Securing Browser Frame Communication, A. Barth, C. Jackson, J. Mitchell
• The Security Architecture of the Chromium Browser, A. Barth, C. Jackson, C. Reis, and the Google Chrome team.

User authentication and session management [PDF, PPT, PDF, PPT]

• Secure Session Management with Cookies for Web Applications, C. Palmer
• Designing and Conducting Phishing Experiments, Finn and Jakobsson, 2007
• Designing an Authentication System: a Dialogue in Four Scenes
• Device Fingerprinting and Fraud Protection Whitepaper, Threat Metrix
• Solving Online Credit Fraud Using Device Identification and Reputation, Iovation

Web site security [PDF, PPT]

• Cross site scripting explained, Amit Klein
• SQL Injection attacks, Chris Anley
• Robust Defenses for Cross-Site Request Forgery, A. Barth, C. Jackson, J. Mitchell

Digital Rights Management [PDF, PPT]

• Hardware-assisted circumvention of self-hashing software tamper resistance, Oorschot et al.