This is the v5.5.3 stable version Note that large sections of the code has been re-written and there are significant changes to the configuration files since v4. V5 is written in c++11 and so to compile it you will need gcc v5.4 or later. (or another complier that supports the full c++11). Note that the target systems may also need an c++11 library update. See notes/NEWIN_v4 for details of the major process model changes incorporated from version 4. REVISED LIST and STORYBOARDING MODEL Version 5 has a completely revised model for defining and using lists. List definition is now separated from list application. Lists are no longer hard-coded, but mapped to a function using a storyboard. Filtering logic flow is simplified and made more consistent. Requests are analysed first and flags set (exception, grey, blocked etc) and once this checking is complete actions are taken. Large sections of duplicate logic has been removed from ConnectionHandler and large sections are now separate functions. Storyboarding is a simple scripting language that maps lists to functions and allows flags to be set. This means that new lists can be added without changing the code, by adding a new list definition and then applying it in a revised storyboard. A different storyboard can be applied to each filtergroup, so if required, each filtergroup can have a different logic flow. Please read notes/V5_list_definition.pdf & notes/V5_Storyboard.pdf for details. TRANSPARENT HTTPS Detects SNI and flags whether traffic is TLS. Currently limited to port 443 traffic on BSD. On Linux multiple ports can be redirected. ICAP SERVER REQMOD and RESPMOD mode supported. See notes/icap. DIRECT UPSTREAM ACCESS I.e. not via proxy. To implement globaly comment out 'proxyip =' in e2guardian.conf. The storyboard action setgodirect can be used within checkrequest functions to enable selected protocols/site/urls to godirect. e.g. to send all connect requests directly add if(connect) setgodirect to a requestchecks function. This can be also useful to by-pass squid for some requests (e.g. os update sites) when squid authentication methods are being used. STORYBOARD TRACING New option 'storyboardtrace' to enable tracing output - for storyboard bug-fixing READABLE THREAD_ID FOR LOGS & DEBUG Most debug and syslog messages are now prefixed with a thread ID as follows:- master: for master thread listen1_proxy: normal proxy listener listen1_thttps: tranparent https listener listen1_icap: icap listener where '1' is index hw10: for http_worker threads where '10' is the thread number log: for logging thread for access.log HTTP/1.1 Support for HTTP v1.1 completed - including Chunked encoding ANTI-VIRUS PLUGINS Anti-virus plugins implemented for proxy, trans and ICAP INCLUDE FILES IN e2guardianf1.conf Filtergroup configuration files may now include other files, enabling a more DRY approach to configuration. So configuration common to several filtergroups can be placed in a file which is included in the filtergroup config file. Syntax is same as list includes - .include OPTION processing Where single options and list defines with the same name are repeated only the last one read will be actioned. This differs from pre-v5 versions where the first single option was actioned and any repeats ignored. This allows the overwriting of single options and re-definition of lists in a structured way. LIST INPUT VIA STDIN This replaces the totalblocklist in previous versions allowing multiple lists to be loaded via stdin. See notes/lists_via_stdin. OPENSSL v1.1 SUPPORT Supports OpenSSL v1.1 Note: Support for pre v1.1 OpenSSL versions is withdrawn in v5.5. % OPTION FOR PHRASE LISTS New % weighting option for phrase lists Code amended so that -Wall compile option gives no warnings BETTER HANDLING OF TRANSPARENT HTTPS REQUESTS Requests without SNI and those not TLS no longer tried for MITM. New option to use original ip & port (currently Linux only) RELATIVE PATHS IN .Includes and list definition The directory of the current .conf (or list file) is taken as the prefix path. LIST DIRECTORY VARIABLE A list directory variable (LISTDIR) can be defined in config files. This works like an exported shell variable and the contents will replace __LISTDIR__ wherever it occurs in a .conf file or list file .Include. This feature will allow the development of templates and easier separation between boilerplate and user configuration. REORGANIZED e2guardian.conf files and list directories. Both example e2guardian.conf files are reorganized into sections and more usage notes added to make it easier for new users to understand what changes they will need. List directory reorganized into sub-directories and READMEs added to help users to know which list(s) to change. The example.group sub-directory should be copied in a groupn directory for each group. Common lists are now in the common sub-directory. As each group's lists are now in a different directory the e2guardianf1.conf files will be able to be copied into a new group .conf with minimal changes. V5.5 NEW LOGGER and DEBUG system for logging, messages and debugging. Thanks to KDGunermann for all his work on this! This replaces the legacy and 'NEWDEBUG' with a unified logging system. Note that the --with-e2debug configure option is replaced by two debug 'levels' --with-debug_high intended for high level debugging (similar to NEWDEBUG, but covering the whole product) and --with-debug_low for low level code debugging See notes/HOWTO_Logger.md and example e2guardian.conf for more details LOG ROTATION New command line option -t for log rotation See notes/rotating_logs for more details SECURE PROXY Secure (TLS) Proxy is now supported. See NETWORK section in e2guardian.conf for more information. OTHER enhancements Made sockets non-blocking so time-out works with MITM - completed but using blocking i/o Reduce unnecessary memory usage on in DataBuffer & DMplugins Remove broken mapauthtoports option/logic. New Bearer over basic token-based auth plugin. Integrate DMplugins with storyboards for plugins. Option for output file for storyboard trace (part of new logging system) Storyboard action by category added alert.log option to enable reporting of key category violations by external email alert and reporting systems 'Auto' MITM certificate generation implimented TESTING - --------- Please prefix all bug reports with "v5.5: " TO DO IN THIS RELEASE --------------------- TODO - for later releases. ----------------------- Create 'output-filter' plug-in system for content modification POST content filtering Implement a "learning" mode, test the rules effects without block Remove duplicate list/storyboard warnings with a DRY system. WISHLIST -------- Adding a 'control' port (something similar to REST API) to pass commands (eg: reload, update lists, etc) and check status Multi-tenancy option. Support for authenticaion helpers to allow ntlm, basic etc auth without using squid proxy. Per-user exceptions/blocks. Per-room blocks. Rate-limiting feature HTTP/2 support (v6) IPv6 support (v6) Philip Pearce 4th April 2025