Proto commits in dreadl0ck/netcap

These 97 commits are when the Protocol Buffers files have changed:

Commit:dcf3741
Author:Philipp Mieden

connection proto updates

The documentation is generated from this commit.

Commit:e313e97
Author:Philipp Mieden

update proto defs for connection flow volume tracking

Commit:e33f53b
Author:Philipp Mieden

modeled alerts, field type updates

Commit:d66ea7f
Author:Philipp Mieden

formatting, renamed field for Ja3 hashmap on profile audit records

Commit:58a7518
Author:Philipp Mieden

move port stats into dedicated structure to avoid alignment issues on ARM

Commit:61a3f39
Author:Philipp Mieden

pop3: save command sequence

Commit:d028984
Author:Philipp Mieden

bootstrapped SMTP stream parser

Commit:3a6b981
Author:Philipp Mieden

removed flow audit records, fixed lookup DHCP fingerprints transforms

Commit:39a4b08
Author:Philipp Mieden

added to contacted ports transform

Commit:56c4a65
Author:Philipp Mieden

add open in disassembler transform, fixed to cookie and to params transforms on host entities, added support to show all http headers and values for a given host

Commit:a086072
Author:Philipp Mieden

use cookie for frontend framework detection, cleanup

Commit:c2d3049
Author:Philipp Mieden

improved build process, added option to compile without dynamically linked dpi libs

Commit:f2ea4c2
Author:Philipp Mieden

fmt

Commit:42e2a56
Author:Philipp Mieden

added a service port to name mapping for service probe matching, to make an educated guess which signatures to try first

Commit:6c5b279
Author:Philipp Mieden

improved port modeling on ipprofiles

Commit:a4dea8c
Author:Philipp Mieden

POP3: move mails into dedicated audit record file and cross reference via IDs

Commit:a13b54b
Author:Philipp Mieden

write IP profile into dedicated audit record file

Commit:a4fe5ef
Author:Philipp Mieden

added command to format the protobuf definitions

Commit:e4051c8
Author:Philipp Mieden

removed Context from structures, added fields directly

Commit:cb91b10
Author:Philipp Mieden

made all timestamps numeric

Commit:0572f97
Author:Philipp Mieden

changed dhcp type for option data field to string

Commit:72bbeaf
Author:Philipp Mieden

store service banners as strings

Commit:0d06a5e
Author:Philipp Mieden

dns audit records: change name fields to type string

Commit:0d78caf
Author:Philipp Mieden

added support to capture payloads for http requests and responses, return an error for ReadHeader(), added packets per second to progress log, added profiling with fgprof, added zeus commands for profiling, move membuffer before the compression step, since pgzip acts as a buffer as well, graphics update, allow to ignore custom encoder init errors in config, rename NoDefrag config Option to DefragIPv4

Commit:17a6f73
Author:Philipp Mieden

refactored service probe matching and added unit tests

Commit:278ebaa
Author:Philipp Mieden

added exploit audit records and refactored queries for bleve databases

Commit:2fc7213
Author:Philipp Mieden

enrich software and vuln maltego entities

Commit:79b4792
Author:Philipp Mieden

maltego iteration, added support for vulnerability audit records

Commit:6b45e2c
Author:Philipp Mieden

added vulnerability audit records, go fmt project

Commit:773a5a0
Author:Philipp Mieden

prepare dhcp fingerprinting, add option to resolve service ips to hostnames via dns lookup with -reverse-dns flag

Commit:a04c767
Author:Philipp Mieden

deduplicate services and store all flows towards a service

Commit:f200bb5
Author:Philipp Mieden

bootstrapped SSH audit records

Commit:78d7490
Author:Philipp Mieden

service audit records: save num bytes seen for server and client

Commit:765857e
Author:Philipp Mieden

bootstrapped credentials audit records and use it to capture http basic auth and pop3 credentials

Commit:04b4cb0
Author:Philipp Mieden

testing and fixes for the service banner regexes, added support for using a different RE engine that supports backtracking

Commit:ee586c0
Author:Philipp Mieden

software audit records: collect flows when updating a record

Commit:9bc4875
Author:Philipp Mieden

updated software fields, preserve full parsed client info for debugging

Commit:2765357
Author:Philipp Mieden

extended service, cleanup and roadmap

Commit:a8c1d2c
Author:Philipp Mieden

added service audit record for banner grabbing

Commit:520dd7a
Author:Philipp Mieden

testing, notes and fixed race in tcp stream reader

Commit:84ef4e1
Author:Philipp Mieden

cleanup

Commit:984f87c
Author:Philipp Mieden

bootstrapped software audit records

Commit:4f833d8
Author:Philipp Mieden

removed layer flows

Commit:c23ff7b
Author:Philipp Mieden

testing and cleanup

Commit:0368d11
Author:Philipp Mieden

added TLSServerHello audit records and renamed TLS records to TLSClientHello

Commit:75c051d
Author:Philipp Mieden

cleanup

Commit:deaa52d
Author:Philipp Mieden

pop3 attachments, version bump

Commit:2ec21cc
Author:Philipp Mieden

HTTP POST parameter extraction, add host to file info, added URL reference to file source field, match file types on the part before the semicolon, to avoid exotic encodings breaking the matching

Commit:2ddc3e0
Author:Philipp Mieden

stream reassembly refactoring: switched to using on assembler per worker with a shared stream pool

Commit:5905d33
Author:Philipp Mieden

implemented GetApplicationCategories and GetApplicationsForCategory

Commit:aeb225f
Author:Philipp Mieden

added POP3 support and first experimental mail extraction plugin for maltego

Commit:49969f5
Author:Philipp Mieden

added experimental POP3 parsing support

Commit:b7c39fd
Author:Philipp Mieden

implemented diameter decoder

Commit:6b5bd9e
Author:Philipp Mieden

added SMTP decoding support

Commit:e49e2bb
Author:Philipp Mieden

modeled http cookies and addeed maltego plugin

Commit:d932559
Author:Philipp Mieden

cleanup

Commit:775a01f
Author:Philipp Mieden

implement CSV generation for DeviceProfile, added File audit record type

Commit:557f8fc
Author:Philipp Mieden

http: added fields for detected content type, added file extraction for http POST, fix files being saved outside of files folder when generating paths for existing targets, improved mapping of content types to file extensions

Commit:1461527
Author:Philipp Mieden

implemented service lookups for ports

Commit:ace955d
Author:Philipp Mieden

added more stats and bootstrapped maltego integration

Commit:697cef0
Author:Philipp Mieden

added nDPI

Commit:2046a94
Author:Philipp Mieden

added cached ja3 lookups, added caching for DNS lookups, added timestamps to profiles

Commit:a1dadea
Author:Philipp Mieden

added descriptive comments about protocols to proto defs

Commit:b109cb9
Author:Philipp Mieden

added type definitions for DeviceProfile and IPProfile

Commit:2e4b029
Author:dreadl0ck

implemented ENIP

Commit:2d4e502
Author:dreadl0ck

implemented support for Common Industrial Protocol

Commit:22132f5
Author:dreadl0ck

updated modbus

Commit:eb7de3f
Author:Philipp Mieden

implemented adding context information to audit records

Commit:bd9d4d1
Author:Philipp Mieden

added comment to http audit record enhanced fields

Commit:3828193
Author:Philipp Mieden

added new HTTP fields

Commit:7eb4b36
Author:Philipp Mieden

added optional payloads for modbusTCP

Commit:abb2158
Author:Philipp Mieden

added gogo proto gode generator for faster protobuf serialization, renamed Size field from several audit records to TotalSize to resolve conflict with generated Size() func

Commit:e5e0b5f
Author:Philipp Mieden

implemented suport for NortelDiscovery

Commit:0313df1
Author:Philipp Mieden

added ContainsPayloads field to Header and Batch structs

Commit:655a29f
Author:Philipp Mieden

added payload flag to preserve payloads for TCP, UDP and USB packets

Commit:f90198a
Author:Philipp Mieden

implemented support for USBRequestBlockSetup

Commit:61390da
Author:Philipp Mieden

added OSPF LSA data to audit records

Commit:b0f689c
Author:Philipp Mieden

implemented merged LayerEncoders to deal with protocols with multiple versions but one gopacket.LayerType (such as OSPF), LSA type definitions, GRE hotfix, LLDI unified string format

Commit:bbb8885
Author:Philipp Mieden

implemented support for CiscoDiscovery protocol

Commit:383b098
Author:Philipp Mieden

implemented support for EAPOL and EAPOLKey

Commit:41700e8
Author:Philipp Mieden

implemented VRRPv2 encoder

Commit:2d3b720
Author:Philipp Mieden

implemented support for EAP protocol

Commit:73253a2
Author:Philipp Mieden

implemented support for FDDI

Commit:67dd59e
Author:Philipp Mieden

implemented support for GRE protocol

Commit:9067552
Author:Philipp Mieden

implemented BFD encoder

Commit:1c653fd
Author:Philipp Mieden

implemented ospf v2 and v3 encoders

Commit:2ffb1e7
Author:Philipp Mieden

implemented modbusTCP encoder

Commit:a323bc3
Author:Philipp Mieden

implemented MPLS encoder

Commit:a573aeb
Author:Philipp Mieden

implemented LCM encoder

Commit:43f7362
Author:Philipp Mieden

implemented support for USB

Commit:7f35c92
Author:Philipp Mieden

VXLAN support

Commit:da5ffe4
Author:Philipp Mieden

added support for ipv6 fragments

Commit:19c2537
Author:Philipp Mieden

added support for geneve protocol

Commit:5b8ed13
Author:Philipp Mieden

added basic ipsec support

Commit:d79fc98
Author:Philipp Mieden

added extensions for tls audit record

Commit:7e8c353
Author:Philipp Mieden

copyright header fix

Commit:0aac60c
Author:Philipp Mieden

hello github