Monday, February 13, 2017

Does Reliable Real Time Detection Demand Prevention?

Chris Sanders started a poll on Twitter asking "Would you rather get a real-time alert with partial context immediately, or a full context alert delayed by 30 mins?" I answered by saying I would prefer full context delayed by 30 minutes. I also replied with the text at left, from my first book The Tao of Network Security Monitoring (2004). It's titled "Real Time Isn't Always the Best Time."

Dustin Webber then asked "if you have [indicators of compromise] IOC that merit 'real-time' notification then you should be in the business of prevention. Right?"

Long ago I decided to not have extended conversations over Twitter, as well as to not try to compress complex thoughts into 140 characters -- hence this post!

There is a difference, in my mind, between high-fidelity matching (using the vernacular from my newest book, The Practice of Network Security Monitoring, 50% off now with code RSAREADING) and prevention.

To Dustin's point, I agree that if it is possible to generate a match (or "alert," etc.) with 100% accuracy (or possibly near 100%, depending on the severity of the problematic event), i.e., with no chance or almost no chance of a false positive, then it is certainly worth seeking a preventive action for that problematic event. To use a phrase from the last decade, "if you can detect it, why can't you prevent it?"

However, there are likely cases where zero- or low-false positive events do not have corresponding preventive actions. Two come to mind.

First, although you can reliably detect a problem, you may not be able to do anything about it. The security team may lack the authority, or technical capability, to implement a preventive action.

Second, although you can reliably detect a problem, you may not want to do anything about it. The security team may desire to instead watch an intruder until such time that containment or incident mitigation is required.

This, then, is my answer to Dustin's question!

Sunday, February 12, 2017

Guest Post: Bamm Visscher on Detection

Yesterday my friend Bamm Visscher published a series of Tweets on detection. I thought readers might like to digest it as a lightly edited blog post. Here, then, is the first ever (as far as I can remember) guest post on TaoSecurity Blog. Enjoy.

When you receive new [threat] intel and apply it in your detection environment, keep in mind all three analysis opportunities: RealTime, Batch, and Hunting.

If your initial intelligence analysis produces high context and quality details, it's a ripe candidate for RealTime detection.

If analysts can quickly and accurately process events generated by the [RealTime] signature, it's a good sign the indicator should be part of RealTime detection. If an analyst struggles to determine if a [RealTime alert] has detected malicious activity, it's likely NOT appropriate for RealTime detection.

If [the threat] intelligence contains limited context and/or details, try leveraging Batch Analysis with scheduled data reports as a better detection technique. Use Batch Analysis to develop better context (both positive and negative hits) to identify better signatures for RealTime detection.

Hunting is the soft science of detection, and best done with a team of diverse skills. Intelligence, content development, and detection should all work together. Don't fear getting skunked on your hunting trips. Keep investing time. The rewards are accumulative. Be sure to pass Hunting rewards into Batch Analysis and RealTime detection operations in the form of improved context.

The biggest mistake organizations make is not placing emphasis outside of RealTime detection, and "shoe-horning" [threat] intelligence into RealTime operations. So called "Atomic Indicators" tend to be the biggest violator of shoe-horning. Atomic indicators are easy to script into signature driven detection devices, but leave an analyst wondering what he is looking at and for.

Do not underestimate the NEGATIVE impact of GOOD [threat] intelligence inappropriately placed into RealTime operations! Mountains of indiscernible events will lead to analyst fatigue and increase the risk of good analyst missing a real incident.

Thursday, February 09, 2017

Bejtlich Books Explained

A reader asked me to explain the differences between two of my books. I decided to write a public response.

If you visit the TaoSecurity Books page, you will see two different types of books. The first type involves books which list me as author or co-author. The second involves books to which I have contributed a chapter, section, or foreword.

This post will only discuss books which list me as author or co-author.

In July 2004 I published The Tao of Network Security Monitoring: Beyond Intrusion Detection. This book was the result of everything I had learned since 1997-98 regarding detecting and responding to intruders, primarily using network-centric means. It is the most complete examination of NSM philosophy available. I am particularly happy with the NSM history appendix. It cites and summarizes influential computer security papers over the four decade history of NSM to that point.

The main problem with the Tao is that certain details of specific software versions are very outdated. Established software like Tcpdump, Argus, and Sguil function much the same way, and the core NSM data types remain timeless. You would not be able to use the Bro chapter with modern Bro versions, for example. Still, I recommend anyone serious about NSM read the Tao.

The introduction describes the Tao using these words:

Part I offers an introduction to Network Security Monitoring, an operational framework for the collection, analysis, and escalation of indications and warnings (I&W) to detect and respond to intrusions.   Part I begins with an analysis of the terms and theory held by NSM practitioners.  The first chapter discusses the security process and defines words like security, risk, and threat.  It also makes assumptions about the intruder and his prey that set the stage for NSM operations.  The second chapter addresses NSM directly, explaining why NSM is not implemented by modern NIDS' alone.  The third chapter focuses on deployment considerations, such as how to access traffic using hubs, taps, SPAN ports, or inline devices.  

Part II begins an exploration of the NSM “product, process, people” triad.  Chapter 4 is a case study called the “reference intrusion model.”  This is an incident explained from the point of view of an omniscient observer.  During this intrusion, the victim collected full content data in two locations.  We will use those two trace files while explaining the tools discussed in Part II.  Following the reference intrusion model, I devote chapters to each of the four types of data which must be collected to perform network security monitoring – full content, session, statistical, and alert data.  Each chapter describes open source tools tested on the FreeBSD operating system and available on other UNIX derivatives.  Part II also includes a look at tools to manipulate and modify traffic.  Featured in Part II are little-discussed NIDS' like Bro and Prelude, and the first true open source NSM suite, Sguil.

Part III continues the NSM triad by discussing processes.  If analysts don’t know how to handle events, they’re likely to ignore them.  I provide best practices in one chapter, and follow with a second chapter explicitly for technical managers.  That material explains how to conduct emergency NSM in an incident response scenario, how to evaluate monitoring vendors, and how to deploy a NSM architecture.

Part IV is intended for analysts and their supervisors.  Entry level and intermediate analysts frequently wonder how to move to the next level of their profession.  I offer some guidance for the five topics with which a security professional should be proficient: weapons and tactics, telecommunications, system administration, scripting and programming, and management and policy.  The next three chapters offer case studies, showing analysts how to apply NSM principles to intrusions and related scenarios.

Part V is the offensive counterpart to the defensive aspects of Parts II, III, and IV.  I discuss how to attack products, processes, and people.  The first chapter examines tools to generate arbitrary packets, manipulate traffic, conduct reconnaissance, and exploit flaws inn Cisco, Solaris, and Microsoft targets.  In a second chapter I rely on my experience performing detection and response to show how intruders attack the mindset and procedures upon which analysts rely.

An epilogue on the future of NSM follows Part V.  The appendices feature several TCP/IP protocol header charts and explanations.   I also wrote an intellectual history of network security, with abstracts of some of the most important papers written during the last twenty-five years.  Please take the time to at least skim this appendix,  You'll see that many of the “revolutionary ideas” heralded in the press were in some cases proposed decades ago.

The Tao lists as 832 pages. I planned to write 10 more chapters, but my publisher and I realized that we needed to get the Tao out the door. ("Real artists ship.") I wanted to address ways to watch traffic leaving the enterprise in order to identify intruders, rather than concentrating on inbound traffic, as was popular in the 1990s and 2000s. Therefore, I wrote Extrusion Detection: Security Monitoring for Internal Intrusions.

I've called the Tao "the Constitution" and Extrusion "the Bill of Rights." These two books were written in 2004-2005, so they are tightly coupled in terms of language and methodology. Because Extrusion is tied more closely with data types, and less with specific software, I think it has aged better in this respect.

The introduction describes Extrusion using these words:

Part I mixes theory with architectural considerations.  Chapter 1 is a recap of the major theories, tools, and techniques from The Tao.  It is important for readers to understand that NSM has a specific technical meaning and that NSM is not the same process as intrusion detection.  Chapter 2 describes the architectural requirements for designing a network best suited to control, detect, and respond to intrusions.  Because this chapter is not written with specific tools in mind, security architects can implement their desired solutions regardless of the remainder of the book.  Chapter 3 explains the theory of extrusion detection and sets the stage for the remainder of the book.  Chapter 4 describes how to gain visibility to internal traffic.  Part I concludes with Chapter 5, original material by Ken Meyers explaining how internal network design can enhance the control and detection of internal threats.

Part II is aimed at security analysts and operators; it is traffic-oriented and requires basic understanding of TCP/IP and packet analysis.  Chapter 6 offers a method of dissecting session and full content data to unearth unauthorized activity.  Chapter 7 offers guidance on responding to intrusions, from a network-centric perspective.  Chapter 8 concludes part III by demonstrating principles of network forensics.

Part III collects case studies of interest to all types of security professionals.  Chapter 9 applies the lessons of Chapter 6 and explains how an internal bot net was discovered using Traffic Threat Assessment.  Chapter 10 features analysis of IRC bot nets, contributed by LURHQ analyst Michael Heiser. 

An epilogue points to future developments.  The first appendix, Appendix A, describes how to install Argus and NetFlow collection tools to capture session data.  Appendix B explains how to install a minimal Snort deployment in an emergency.  Appendix C, by Tenable Network Security founder Ron Gula, examines the variety of host and vulnerability enumeration techniques available in commercial and open source tools.  The book concludes with Appendix D, where Red Cliff Consulting expert Rohyt Belani offers guidance on internal host enumeration using open source tools.

At the same time I was writing Tao and Extrusion, I was collaborating with my friends and colleagues Keith Jones and Curtis Rose on a third book, Real Digital Forensics: Computer Security and Incident Response. This was a ground-breaking effort, published in October 2005. What made this book so interesting is that Keith, Curtis and I created workstations running live software, compromised each one, and then provided forensic evidence for readers on a companion DVD. 

This had never been done in book form, and after surviving the process we understood why! The legal issues alone were enough to almost make us abandon the effort. Microsoft did not want us to "distribute" a forensic image of a Windows system, so we had to zero out key Windows binaries to satisfy their lawyers. 

The primary weakness of the book in 2017 is that operating systems have evolved, and many more forensics books have been written. It continues to be an interesting exercise to examine the forensic practices advocated by the book to see how they apply to more modern situations.

This review of the book includes a summary of the contents:

• Live incident response (collecting and analyzing volatile and nonvolatile data; 72 pp.)
• Collecting and analyzing network-based data (live network surveillance and data analysis; 87 pp.)
• Forensic duplication of various devices using commercial and open source tools (43 pp.)
• Basic media analysis (deleted data recovery, metadata, hash analysis, “carving”/signature analysis, keyword searching, web browsing history, email, and registry analyses; 96 pp.)
• Unknown tool/binary analysis (180 pp.)
• Creating the “ultimate response CD” (response toolkit creation; 31 pp.)
• Mobile device and removable media forensics (79 pp.)
• On-line-based forensics (tracing emails and domain name ownership; 30 pp.)
• Introduction to Perl scripting (12 pp.)

After those three titles, I was done with writing for a while. However, in 2012 I taught a class for Black Hat in Abu Dhabi. I realized many of the students lacked the fundamental understanding of how networks operated and how network security monitoring could help them detect and respond to intrusions. I decided to write a book that would explain NSM from the ground up. While I assumed the reader would have familiarity with computing and some security concepts, I did not try to write the book for existing security experts.

The result was The Practice of Network Security Monitoring: Understanding Incident Detection and Response. If you are new to NSM, this is the first book you should buy and read. It is a very popular title and it distills my philosophy and practice into the most digestible form, in 376 pages.

The main drawback of the book is the integration of Security Onion coverage. SO is a wonderful open source suite, partly because it is kept so current. That makes it difficult for a print book to track changes in the software installation and configuration options. While you can still use PNSM to install and use SO, you are better off relying on Doug Burks' excellent online documentation. 

PNSM is an awesome resource for learning how to use SO and other tools to detect and respond to intrusions. I am particularly pleased with chapter 9, on NSM operations. It is a joke that I often tell people to "read chapter 9" when anyone asks me about CIRTs.

The introduction describes PNSM using these words:

Part I, “Getting Started,” introduces NSM and how to think about sensor placement.

• Chapter 1, “Network Security Monitoring Rationale,” explains why NSM matters, to help you gain the support needed to deploy NSM in your environment.
• Chapter 2, “Collecting Network Traffic: Access, Storage, and Management,”addresses the challenges and solutions surrounding physical access to network traffic.

Part II, “Security Onion Deployment,” focuses on installing SO on hardware and configuring SO effectively.

• Chapter 3, “Stand-alone NSM Deployment and Installation,” introduces SO and explains how to install the software on spare hardware to gain initial NSM capability at low or no cost.
• Chapter 4, “Distributed Deployment,” extends Chapter 3 to describe how to install a dispersed SO system.
• Chapter 5, “SO Platform Housekeeping,” discusses maintenance activities for keeping your SO installation running smoothly. 

Part III, “Tools,” describes key software shipped with SO and how to use these applications.

• Chapter 6, “Command Line Packet Analysis Tools,” explains the key features of Tcpdump, Tshark, Dumpcap, and Argus in SO.
• Chapter 7, “Graphical Packet Analysis Tools,” adds GUI-based software to the mix, describing Wireshark, Xplico, and NetworkMiner.
• Chapter 8, “NSM Consoles,” shows how NSM suites, like Sguil, Squert, Snorby, and ELSA, enable detection and response workflows.

Part IV, “NSM in Action,” discusses how to use NSM processes and data to detect and respond to intrusions.

• Chapter 9, “NSM Operations,” shares my experience building and leading a global computer incident response team (CIRT).
• Chapter 10, “Server-side Compromise,” is the first NSM case study, wherein you’ll learn how to apply NSM principles to identify and validate the compromise of an Internet-facing application.
• Chapter 11, “Client-side Compromise,” is the second NSM case study, offering an example of a user being victimized by a client-side attack.
• Chapter 12, “Extending SO,” concludes the main text with coverage of tools and techniques to expand SO’s capabilities.
• Chapter 13, “Proxies and Checksums,” concludes the main text by addressing two challenges to conducting NSM.

The Conclusion offers a few thoughts on the future of NSM, especially with respect to cloud environments. 

The Appendix, “SO Scripts and Configuration,” includes information from SO developer Doug Burks on core SO configuration files and control scripts.

I hope this post helps explain the different books I've written, as well as their applicability to modern security scenarios.

Tuesday, January 31, 2017

Meeting Cliff Stoll

Today I had the chance to meet the man who unintentionally invented the modern digital forensics practice, Cliff Stoll. In 1989 he published a book about his 1986-87 detection and response against KGB-backed spies who hacked his lab and hundreds of government, military, and university computers. I read his book in high school and it later inspired my military and private computer security services. Cliff was kind enough to take a photo with me today at the SANS Institute Cyber Threat Intelligence Summit in Virginia.

Sunday, December 18, 2016

Check Out My TeePublic Designs

Over the years fans of this blog have asked if I would consider selling merchandise with the TaoSecurity logo. When I taught classes for TaoSecurity from 2005-2007 I designed T-shirts for my students and provided them as part of the registration package. This weekend I decided to exercise my creative side by uploading some designs to TeePublic.

TeePublic offers clothing along with mugs, phone cases, notebooks, and other items.

Two are based on the TaoSecurity logo. One includes the entire logo, along with the company motto of "The Way of Digital Security." The second is a close-up of the TaoSecurity S, which is a modified yin-yang symbol.

Two other designs are inspired by network security monitoring. One is a 1989-era map of MilNet, the United States' military network. This image is found in many places on the Internet, and I used it previously in my classes. The second is a close-up of a switch and router from the TaoSecurity labs. I used this equipment to create packet captures for teaching network security monitoring.

I hope you like these designs. I am particularly partial to the TaoSecurity Logo mug, the TaoSecurity S Logo Mug, and TaoSecurity S Logo t-shirt.

Let me know what you think via comments here.

Update 28 Dec 2016:

Check out the MilNet mug!


Tuesday, October 18, 2016

Five Ways That Good Guys Share More Than Bad Guys

It takes a lot for me to write a cybersecurity blog post these days. I spend most of my writing time working on my PhD. Articles like Nothing Brings Banks Together Like A Good Hack drive me up the wall, however, and a Tweet rant is insufficient. What fired me up, you might ask? Please read the following excerpt:

[Troels] Oerting, with no small dose of grudging admiration, says his adversaries excel at something that can’t be addressed with deep pockets or killer software: They’re superb networkers. “The organized crime groups in cyber are sharing much better than we are at the moment,” says Oerting, a Dane with a square jaw and the watchful eyes of a cop who’s investigated the underworld for 35 years. “They are sharing methodologies, knowledge, tools, practices—what works and what doesn’t.”

Statements like these are regularly submitted without evidence. In response, I provide five sources of evidence why organized crime groups do not share more than defenders.

1. Solution providers share. Both commercial and not-for-profit solution providers share enormous amounts of information on the security landscape. Some of it is free, and some of it is sold as products or consulting. Thousands of security companies and not-for-profit providers compete for your attention, producing white papers, Webinars, and other resources. You might argue that all of them claim to be the answer to your problem. However, this situation is infinitely better than the 1980s and early 1990s. Back then, hardly any solutions, or even security companies and organizations, existed at all.

Criminal solution providers share, but they do so by selling their wares. This is true for the open world as well, but the volume of the open world is orders of magnitude greater.

2. Government agencies share. My fellow Americans, has your organization you been visited by the FBI? Federal agents notified more than 3,000 U.S. companies [in 2013] that their computer systems had been hacked. The agents didn't just walk in, drop a letter, and leave. If a relationship did not exist previously, it will now be developed.

Beyond third party breach notifications, agencies such as NIST, DHS, and others regularly share information with organizations. They may not share as much as we would like, but again, historical perspective reveals great progress.

3. Books, articles, and social media share. The amount of readable material on security is astounding. Again, in the late 1980s and early 1990s hardly any books or articles were available. Now, thousands of resources exist, with new material from publishers like No Starch arriving monthly. Where are the books written by the underground?

4. Security conferences share. You could spend every week of the year at a security conference. If you happen to miss a talk, it's likely the incomparable Iron Geek recorded it. Does the underground offer similar opportunities?

5. Private groups and limited information exchange groups share. A final category of defender sharing takes place in more controlled settings. These involve well-established Information Sharing and Analysis Centers (ISACs), developing Information Sharing and Analysis Organizations (ISAOs), and private mailing lists and forums with limited membership. These could possibly be the closest analogue to the much-esteemed underground. Even if you disregard points 1-4 above, the quality of information shared in this final category absolutely equals, if not exceeds, anything you would find in the criminal world.

If you disagree with this analysis, and continue to lament that bad guys share more than the good guys, what evidence can you provide?

Monday, June 27, 2016

Updated PhD Thesis Title

Yesterday I posted Latest PhD Thesis Title and Abstract. One of my colleagues Ben Buchanan subsequently contacted me via Twitter and we exchanged a few messages. He prompted me to think about the title.

Later I ruminated on the title of a recent book by my advisor, Dr. Thomas Rid. He wrote Cyber War Will Not Take Place. One of the best parts of the book is the title. In six words you get his argument as succinctly as possible. (It could be five words if you pushed "cyber" and "war" together, but the thought alone makes me cringe, in the age of cyber-everything.)

I wondered if I could transform my latest attempt at a thesis title into something that captured my argument in a succinct form.

I thought about the obsession of the majority of the information security community on the tool and tactics level of war. Too many technicians think about security as a single-exchange contest between an attacker and a defender, like a duel.

That reminded me of a problem I have with Carl von Clausewitz's definition of war.

We shall not enter into any of the abstruse definitions of war used by publicists. We shall keep to the element of the thing itself, to a duel. War is nothing but a duel on an extensive scale.

- On War, Chapter 1

Clausewitz continues by mentioning "the countless number of duels which make up a war," and then makes his famous statement that "War therefore is an act of violence to compel our opponent to fulfill our will." However, I've never liked the tactically-minded idea that war is a "duel."

This concept, plus the goal to deliver a compact argument, inspired me to revise my thesis title and subtitle to the following:

Campaigns, Not Duels: The Operational Art of Cyber Intrusions

In the first three words I deliver my argument, and in the subtitle I provide context by including my key perspective ("operational art"), environment ("cyber," yes, a little part of me is dying, but it's a keyword), and "intrusions."

When I publish the thesis as a book in 2018, I hope to use the same words in the book title.