ViewVC logotype

Contents of /main/rutil/test/runtests.sh

Parent Directory Parent Directory | Revision Log Revision Log

Revision 9633 - (show annotations) (download)
Wed May 2 19:27:38 2012 UTC (7 years, 6 months ago) by sgodin
File MIME type: text/plain
File size: 766 byte(s)
-merge work from branches/b-repro-enh-20120205

-up repro version number 0.9
-added new Settings page to repro web interface to show current command line / files settings in use
 - will also display some some low level stack info and congestion stats (if enabled)
-added new config setting StatisticsLogInterval to specify how often statistics are dumped to 
 the log files
-cleaned up the WebAdmin constructor to take a Proxy object
-added new KeyValueStore class for highly efficient generic storage
 - keys are simple integer indexes into a vector and must be allocated 
   before they are used
 - currently supports storage and retrieval of the following types:
   Data, bool, short, unsigned short, int, unsigned int, and UInt64
-Converted ConfigParse storage from std::multimap to HashMultiMap
-removed print in TcpConnection that could end up printing garbage at the end of 
 messages that are not null terminated
-added KeyValueStore to three strategic locations in repro, allowing custom Processors 
 (Monkeys, Lemurs and Baboons) to store state scoped as follows:
     -Global Proxy Scope - Proxy::getKeyValueStore
     -Request Scope - RequestContext::getKeyValueStore
     -Target Scope - Target::getKeyValueStore
 Before this storage can be used you must statically allocate a storage key.
 See mFromTrustedNodeKey use in the IsTrustedNode class for an example.
-major changes to how repro is started up, to allow easier additions of
 custom startup logic, such as adding custom Processors (Monkeys, Lemurs and 
 Baboons) to the default Processor chains
  - almost all logic that was in repro.cxx has been moved out to a new class
    (ReproRunner), and split into smaller virtual methods that can be overridden
  - see comments at the top of repro.cxx for an example of how to add custom
-added option to enable some basic P-Asserted-Identity header handling
 - After auth is successful 
   - if P-Perferred-Identity header is present remove it
   - if no P-Asserted-Identity header is present, then add one
 - Removal of P-Asserted-Identity if Privacy header is set to "id"
   Note: Since we have no better mechanism to determine if destination is 
         trusted or not we assume that all destinations outside our domain 
         are not-trusted and will remove the P-Asserted-Identity header
-modified how ;lr param on routes are processed - moved logic from StaticRoute 
 to ResponseContext - this slightly changes the behaviour
 - previous implementation used to stop the processing chain as soon
   as the first matching route with a ;lr is found, new implementation
   does not stop Target processing if a ;lr is present
 - Multiple targets can now have a ;lr on them and be processed
   properly.  The ;lr parameter is now examined in 
   ResponseContext::beginClientTransaction, and if present the RequestUri
   is not altered and a Route header is added instead
-fixed a duplicate removeContact call in OutboundTargetHandler (a merge conflict
-avoid unnecessary iteration through target list in StaticRoute when try to determine
 if auth is required
-added two new constructors to QValueTarget to make is easier to form targets from 
 a NameAddr or Uri only
-removed some unused code in ResponseContext:  addOutboundBatch and mOutboundMap
-renamed Target::targetPtrCompare to Target::priorityMetricCompare to be more descriptive
-implement proper q-value processing of contacts in a redirect response

-fix for a long standing issue in repro that started in rev6794, where
 repro can be over protective and issue 403 responses for legitimate 
 mid-dialog requests.  The issue occurs when a repro domain user forms
 a dialog with a user in another external domain.  Any mid-dialog
 requests coming from the external domain would get 403'd.  This 
 was due to the logic in the AmIResponsible monkey, and the fact that 
 such requests have the repro endpoints contact address in the 
 RequestUri (typically the endpoints IP address), so not belonging
 to repro's domain, and the From user is not being from repro's domain.

 The original code to only do these checks for out of dialog requests
 was reintroduced with the following comments:

         // only perform relay check for out-of-dialog requests
         // !bwc! Um, then all anyone has to do to get us to be their relay
         //       is throw in a spurious to-tag...
         //       This smells funny. I am commenting it out.
         // .slg. Putting code back in and clarifying the funny smell.....
         //       We only want to do this check for out of dialog requests, since 
         //       mid-dialog requests could be 403'd otherwise.  Consider
         //       an INVITE request from a repro domain user to a user in 
         //       another domain.  The resulting ACK/200, BYE or any other
         //       mid-dialog request coming from the remote domain, will contain
         //       the repro users contact address in the RequestUri and a 
         //       foreign domain in the from header.  We want to ensure these
         //       requests are not 403'd.  Byron's comment about an endpoint getting
         //       us to relay by placing a spurious to tag in the request still 
         //       stands. Perhaps we ought to be checking the To header domain in 
         //       this case - however that is also weak, since the To header is not
         //       used in routing and easily be set to a URI in our domain to trick
         //       repro into forwarding.  Note:  From header domain checking is
         //       stronger than To header domain checking, since if the domain is 
         //       ours, then it must pass Digest Authentication (at least for non 
         //       ACK and BYE requests).

-add ability for a repro admin to add manual / permanent registrations - such
 manually added registrations are persisted to the database, and loaded at startup
 - Manual registrations can be added on the Registration Web Page
-add display of registered contact's QValue on registrations web page
-added ability to specify a Path on new static registration feature
-cleanup some old hacks now that we have the ability to manually add registrations
  - ParallelForkStaticRoutes no longer combines StaticRoutes Targets and LocationServer
  - Static Routes are no longer added as QValueTargets, they are now added as simple Targets.
    So they are no longer susceptible to the various QValue Settings - ie. QValueMsBeforeCancel.
  - Added new ContinueProcessingAfterRoutesFound setting:
    By default (false) we will stop looking for more Targets if we have found
    matching routes.  Setting this value to true will allow the LocationServer Monkey
    to run after StaticRoutes have been found.  In this case the matching
    StaticRoutes become fallback targets, processed only after all location server 
    Targets fail.

-added new Helper method:  Tuple getClientPublicAddress(const SipMessage& request)
  - look at Via headers, and finds the first public IP address closest to the sending
-added loopback address checking to Tuple::isPrivateAddress
-modified ServerRegistration:
 - Ensure that ContactInstanceRecord::mReceivedFrom is always populated - not just 
   in outbound use cases - added a new flag to indicate when flow routing is required
 - Added a new mPublicAddress flag member to ContactInstanceRecord - this will assist
   with an upcoming feature to do geo proximity routing
-added new ContactInstanceRecord::mPublicAddress to regsync process
-removed a chatty log line in DialogSetId
-used Symbols::COMMA in a few spots it wasn't being used in
-removed unimplemented targetCount method on ResponseContext
-renamed Helper::isSenderBehindNAT to isClientBehindNAT for consistency
-ensure we only skip the Monkey chain in StaticRoute if we actual found routes
-added missing license block to ForkControlMessage
-adding new Baboon:  GeoProximityTargetSorter
 If enabled, then this baboon can post-process the target list.  
 This includes targets from the StaticRoute monkey and/or targets
 from the LocationServer monkey.  Requests that meet the filter 
 criteria will have their Target list, flatened (serialized) and
 ordered based on the proximity of the target to the client sending
 the request.  Proximity is determined by looking for a 
 x-repro-geolocation="<latitude>,<longitude>" parameter on the Contact
 header of a received request, or the Contact headers of Registration
 requests.  If this parameter is not found, then this processor will
 attempt to determine the public IP address closest to the client or
 target and use the MaxMind Geo IP library to lookup the geo location.
-requires floating point support for geo proximity calculations
-resip TCP transports can crash repro on uncaught exception - if garbage is received
 on the socket, and there is no Content-Length header, then SipMessage::Exception can
 throw, and it was not caught with the existing ParseException catch handler.  Changed
 to catch BaseException instead.
-removed unsafe logging statement in DumTimeout - if DUM and stack are in different threads, then
 crash could occur
-enable Inserter functionality for collections that store pointers
 to items.  Use InsertP instead of Inserter for such collections.
 For example:  
 std::vector<std::string*> v;
 ... populate vector ...
 std::cout << InserterP(v) << std::endl;
-statictize strings in Inserter/InserterP
-add InserterP test cases to rutil/test/testInserter.cxx
-cleaned up Processor and ProcessorChain classes
 - added mName property in anticipation of a future capability to define processor 
   chains in the configuration file
 - simplified operator<< for processors, to use Name
-modified some Inserter uses to use new InserterP function
-allow custom repro implementations to add themselves to the RegistrarHandler so that
 registration messages can be processed and reacted to
-added a singleResultQuery API to MySqlDb
-series of interfaces to make is easier to implement new Asynchronous Monkey's / Processors that utilize a common thread pool
-removed getTransactionId from ForkControlMessage - method exists on base class, so it's not needed
-removed getTransactionId and tid() from UserInfoMessage - getTransactionId method exists on base class, so they are not needed
-made Worker and abstract base class
-modified ConfigParser to throw a resip style exception when failing to open configuration file
-minor cleanups to ConfigParser configuration file parsing routine

-added new RequestFilter monkey
 - allows user to configure conditions under which an inbound request should be rejected or not
 - allows two regular expression conditions that can be applied to any SIP message header:
   this includes the request-line, standard SIP headers and custom SIP headers.  If a header
   that can appear multiple time is specified, then each instance of the header is checked.
 - When conditions are met, allows the action carried out to be defined:
   - Accept - accepts this request and stops further processing in Request Filter monkey 
   - Reject - rejects this request with the provided SIP status code and reason text
   - SQL query - only available when MySQL support is compiled in - runs an arbitrary stored
     procedure or query, using replacement strings from the 2 condition regular expressions
     - query must return an empty string or "0" to instruct repro to Accept the request, or a 
       string containing "<SIP Reject Status Code>[, <SIP Reject Reason>]" to Reject the request
     - using the repro configuration file the SQL Query can be configured to operate on a
       completely different my SQL instance/server than the repro configuration
   - Filters are defined in the HTTP web interface via new Add Filter, Edit Filter and Show
     Filters web pages.  There is an ability to test the condition regular expressions from 
     the web page as well.
   - other Monkey settings are configured in the repro configuration file or via command line:
     DisableRequestFilterProcessor, RequestFilterDefaultNoMatchBehavior, 
     RequestFilterDefaultDBErrorBehavior, RequestFilterMySQLServer (and other mySQL related settings) 
   - can be used to implement a User Blocking functionality - ie. calls and instant messages 
     from user X to user Y should always be blocked, because user X is in user Y's block list
   - introduced new FilterStore configuration database table to store the Filters configured
     on the web pages

-added AsyncProcessorDispatcher / thread pool to repro that be shared by all AsyncProcessors
 - currently only used by new RequestFilter monkey
-cleaned up some implementation in AbstractDb to remove some code duplication
-catch config file parse, and missing file errors (exceptions) at repro startup
  - stop webpage from being able to add two routes with the same Key
  - optimized data fetch for displaying routes on web page
-Repro Web Interface
 - added new Add Filter, Edit Filter and Show Filters pages
 - made use of HTML tables consistent across all pages
 - made table backgrounds white to improved appearance
 - added title to right hand side pane
 - cleaned up formatting on many pages
 - ordered methods in source to match ordering on web page 
 - added warning to Domains page, that repro must be restarted

-added version information to startup logs and to display on web interface
-added bottom footer on web interface with link to www.resiprocate.org
-added new RequestFilter files to all Visual Studio projects and linux Makefile
-upped repro version to 0.9

-fix a bug in repro web interface, where fragmented HTTP messages were not being handled correctly
-fixed possible assert if a transport error is seen after trying to send an ACK message

-added optional MessageSilo support to repro
 - stores IM's (ie. SIP MESSAGE requests) for offline users
 - replays messages to users when they register (ie. come back online)
 - records are persisted to a database table, so they survive shutdowns
 - configurable filters exist for DestUri, MimeType, MessageBody size

-moved CommandServer from repro to reprolib
-added restart command to reprocmd - allows restarting repro to apply configuration changes while leaving 
 the in memory registration database in tact
-propagate db insert/update failures to callers - web interface now shows errors if record fails to update in db
-remove unused AbstractDb API's: writeRoute and writeFilter
-added ability for tables to have non-unique keys (ie. duplicate records)
-modified WorkerThread to support work that does not require a response to be queued back to the stack
-added database transaction support to BerekelyDb and MySQL implementations
-reorganized repro runner startup ordering so that DUM TU is started before Proxy and added new 
 virtual createDatastore method - in order to facility new Silo Monkey startup that requires access to 
 the Registrar object
-make RequestFilter monkey options - use config flag
-allow a DateCategory to be created from time_t

-build fixes for linux, add --with-geoip option to autotools configure
-added --with-popt to configure so that test projects can be build easily with popt support
-added autodetection of epoll support on unix and automatic addition of HAVE_EPOLL define

-added ability for repro to report a 404 error when attempting to reach a user that does not exist
 - previously repro would always send a 480 response when attempting to reach an AOR that wasn't
-removed unused SipStack parameter to DigestAuthenticator monkey

-allow reprocmd to run defaulting to for server connection

-added secondary database support to BerkeleyDb - allows tables with a secondary index
 - my sql implementation of AVPs with two indexes was added as well
-message silo records can now be deleted by referencing a unique key (ie. timestamp+tid)
 - transaction support is no longer required
-optimized data copies when reading records from BerkeleyDb
-increase MySQL AVP table size for value field from 1024 to 4096
-implemented logic to periodically cleanup Silo records that have expired

1 #!/bin/bash
3 function die() {
4 echo "$1" >&2;
5 exit 1;
6 }
8 failed=0;
11 for i in \
12 testCoders \
13 testCountStream \
14 testData \
15 testDataPerformance \
16 testDataStream \
17 testDnsUtil \
18 testFifo \
19 testFileSystem \
20 testInserter \
21 testIntrusiveList \
22 testKeyValueStore \
23 testLogger \
24 testMD5Stream \
25 testRandomHex \
26 testSHA1Stream \
27 testParseBuffer \
28 testThreadIf \
29 testXMLCursor;
30 do
31 if test ! -x $i; then
32 echo "$i: test does not exist" >&2;
33 else
34 ./$i >$i.log 2>&1;
35 if test $? -ne 0; then
36 die "$i: test failed";
37 failed=`expr ${failed} + 1`;
38 fi;
39 echo "$i: test passed";
40 fi;
41 done;
43 if test ${failed} -ne 0; then
44 echo "${failed} tests failed" >&2;
45 exit 1;
46 fi;
48 # testMem \


Name Value
svn:eol-style native
svn:executable *
svn:mime-type text/plain

webmaster AT resiprocate DOT org
ViewVC Help
Powered by ViewVC 1.1.27