Flow-Portscan module documentation:
Initial Discussions:
Marc Norton
Dan Roelker
Chris Green
Implementation:
Chris Green everything but sfxhash
Documentation to-dos:
- explain the time domains
- explain the scoring domains
Documentation last updated: 2003-09-22
This is module is designed to detect rapid portscans based off flow
creation in the flow preprocessors. The goals is to catch one->many
hosts and one->many ports scans.
The flow preprocessor to portscan recognizer is taken from
experience with spp_conversation/portscan2 by Jason Larsen & Jed
Haile and ipaudit by Jon Rifkin.
This subsystem became a bit more complicated than originally
intended but it does a good job of mitigating false positives from
devices such as squid proxies. The new design is also a lot more
memory consistent than portscan1 or 2. It also ignores single port
syn floods as they are a DoS, not a portscan.
Memory requirements should be way down from portscan2 architecture
though but there's slightly less information saved off. The new
architecture operates similarly to a ring buffer. When a scanner
has not been active in a long time, it's only reclaimed when there
is no more memory to use.
All of the prior methods for portscan detection in snort are
deprecated and will be removed in the near future. If you have
custom code against conversation or one of the portscan
preprocessors, consider making it a module in flow or portscan.
Basic components:
2 Scoreboards ( One Talker, One Scanner )
Scoreboards contain information regarding timescales for a single
IP address. There are two scoreboards, one for talkers (nodes
that are active on your network) and one for scanners (nodes that
have talked to a previously unknown port in your
server-watch-net)
1 Uniqueness tracker
The uniqueness tracker is used to determine if this connection
should count as something "new" for a particular IP. It checks
if a connection is a new type of connection for a Source IP by
disregarding the source port.
Any change in (SIP,DIP,IP_PROTO,DPORT) indicates a new unique
connection and will be processed further for the server
statistics table and scoring.
This keeps things like a web page with 15 images from
rapidly increasing point scores with lots of accesses to the same
web server.
1 Server Statistics Tracker
This is used to track flows destined to the "server-watchnet"
and keep "hitcounts" on the number of times a particular service
has been requested with unique requests since snort has started.
This hitcount is tracked by (DIP,DPORT,PROTOCOL).
If a service is very popular, we can make connections to it be
ignored for scoring by comparing the hitcount to the
"server-ignore-limit". If we have more requests to this service
than the server-ignore-limit, then we will completely ignore this
service. Similarly, the "server-scanner-limit" controls if a
request to a service counts as scanner points or as talker points.
If a request to a service is not in the server-watch-net, it will
count as talker points.
Caveat:
This does not perform validation that the service is connected
correctly so it is possible while learning that someone floods the
table with unique connections that it is possible to have
something become a service that you do not wish to be a service.
It's generally assumed that the learning time will occur at a time
where traffic is "typical". Future versions of snort should allow
this state to be saved and modifiable.
If this caveat is a concern in your environment, do not set a server
watchnet and rely only on talker scores.
Module Overview:
1) flow-portscan receives a new flow message from the flow module
2) The uniqueness tracker determines if message is a new type of
flow by looking for changes in (SIP,DIP,IP_PROTO,DPORT). If this is
not unique, and the TCP flags are normal, exit out.
3) If this connection is to an Destination IP in the server-watchnet:
During the "server-learning-time", it increments the hitcounts
for service popularity.
If it's otherwise just get the stored hitcount. If the hitcount
is greater than the server-ignore-limit, exit out. If it's less
than the server-scanner-limit, mark the incremented points as
scanner points.
4) A connection is marked as either a talker or a scanner by step 3.
There are 4 time scales; 2 each for the IP Scanner and IP Talker.
The fixed timescales detect N events in M seconds. This is the
typical type of portscan alert.
The sliding timescales adjust the "score reset point" on each
event after the first. This adjusts the side of the window we're
detecting portscan events in by taking
end = end + ((end - start) * sliding-scale-factor)
Each time scale has it's own point tally that is incremented per
new flow. Each set of points only touches either the
talker-fixed-score and talker-sliding-score
OR
scanner-fixed-score and scanner-sliding-score
5) Evaluate the score against individual thresholds, either talker
or scanner.
if(fixed_limit <= fixed_score)
generate_alert()
flow-portscan options:
General Note: higher row counts will take more memory away from the
memory caps for a specific subsystem. In the snort output, this is
referred to as "overhead bytes" and the percentage of overhead
encountered will be shown. Higher row counts provide a larger hash
table to minimize collisions and have a faster overall processing time
at the expense of memory. The hash tables themselves use a
pseudorandom hardening salt that is picked at initialization time.
scoreboard-memcap-talker <bytes>
Number of bytes to use for the talker table
scoreboard-rows-talker <count>
Number of rows to use for the talker table
scoreboard-rows-scanner <count>
Number of rows to use for the scanner table
scoreboard-memcap-scanner <bytes>
Number of bytes to use for the scanner table
scanner-fixed-threshold <integer>
Number of points that a scanner must accumulate in the
scanner-fixed-window time range. Set to 0 to disable this type of
alert.
scanner-sliding-threshold <integer>
Number of points that a scanner must accumulate in
scanner-sliding-window time range. Set to 0 to disable this type of
alert.
scanner-fixed-window <integer>
How many seconds we should go before resetting the fixed scanner score
scanner-sliding-window <integer>
How many seconds we should go before resetting the sliding scanner score
scanner-sliding-scale-factor <float>
How much to increase the sliding window by each time we get a new
sliding scanner entry. It's current size + (<scale factor> * current_size)
talker-fixed-threshold <integer>
Number of points that a scanner must accumulate in
talker-fixed-window time range. Set to 0 to disable this type of
alert.
talker-sliding-threshold <integer>
Number of points that a scanner must accumulate in
talker-sliding-window time range. Set to 0 to disable this type of
alert.
talker-fixed-window <integer>
How many seconds we should go before resetting the fixed talker score
talker-sliding-window <integer>
How many seconds we should go before resetting the sliding talker score
talker-sliding-scale-factor <float>
How much to increase the sliding window by each time we get a new
sliding talker entry. It's current size + (<scale factor> * current_size)
unique-memcap <bytes>
How many bytes to allocate to the uniqueness tracker. The more
memory given, the less that connections to a busy server will appear
as a scan target on a popular service.
unique-rows <integer>
How many rows to allocate for the uniqueness tracker.
server-memcap <bytes>
How many bytes to allocate for server learning
server-rows <integer>
How many rows to allocate for server learning
server-watchnet <ip list in snort notation>
The IP list of what machines to learn services on. Busy servers
should be placed here to help the portscan detector learn what
services are requested on the network.
src-ignore-net <ip list in snort notation>
The IP list of what Source IP's to ignore.
dst-ignore-net <ip list in snort notation>
The IP list of what Destination IP's to ignore.
tcp-penalties <on|off>
If this is enabled, when a new tcp flow enters the portscan
detection set, check the TCP flags for non-standard session
initiators and assign penalty points for odd combinations such as
SYN+FIN
Flag mapping:
SYN or SYN+ECN bits == base_score ( defaults to 1 point )
SYN+FIN+TH_ACK and anything else == 5 points
SYN+FIN and anything else without ack == 3 points
Anything else == 2 points
server-learning-time <seconds>
How many seconds we should keep increment hitcounts of services on
IP's in the server-watchnet
server-ignore-limit <hit count>
How many requests a port on an IP in the server-watchnet must see
before it is ignored for the purposes of portscans.
server-scanner-limit <hit count>
How many requests a port on an IP in the server-watchnet must see
before it is is treated as a talker rather than a scanner. This is
a minimum number of requests that must be seen during the
server-learning-time for the flow to be treated as a talker
connection rather than as a scanner connection.
alert-mode <once|all>
In once mode, alert only on the first time we get a scan entry hit.
This dramatically reduces clutter because the scan alert in the
first place tells one to look for other event types
.
On All, alert each time the score increases beyond a threshold.
output-mode <msg|pktkludge>
msg - a variable text message with the scores included
pktkludge - generate a fake pkt and use the Logging output system
dumpall 1
When snort is exiting, dump the entire contents of the server table,
the uniqueness tracker table, and the scoreboard entries. This is
' useful if you suspect an underlying bug in the algorithms used or if
you would just like to see what it has learned.
Example Configuration:
preprocessor flow: stats_interval 0
preprocessor flow-portscan: \
server-watchnet [10.0.0.0/8] \
unique-memcap 5000000 unique-rows 50000 \
tcp-penalties on \
server-scanner-limit 50 \
alert-mode all \
output-mode msg \
server-learning-time 3600