2005-03-18 jrandom
* Minor tweak to the timestamper to help reduce small skews
* Adjust the stats published to include only the relevent ones
* Only show the currently used speed calculation on the profile page
* Allow the full max # resends to be sent, rather than piggybacking the
RESET packet along side the final resend (duh)
* Add irc.postman.i2p to the default list of IRC servers for new installs
* Drop support for routers running 0.5 or 0.5.0.1 while maintaining
backwards compatability for users running 0.5.0.2.
* Update the old speed calculator and associated profile data points to
use a non-tiered moving average of the tunnel test time, avoiding the
freshness issues of the old tiered speed stats.
* Explicitly synchronize all of the methods on the PRNG, rather than just
the feeder methods (sun and kaffe only need the feeder, but it seems ibm
needs all of them synchronized).
* Properly use the tunnel tests as part of the profile stats.
* Don't flood the jobqueue with sequential persist profile tasks, but
instead, inject a brief scheduling delay between them.
* Reduce the TCP connection establishment timeout to 20s (which is still
absurdly excessive)
* Reduced the max resend delay to 30s so we can get some resends in when
dealing with client apps that hang up early (e.g. wget)
* Added more alternative socketManager factories (good call aum!)
2005-03-11 jrandom
* Rather than the fixed resend timeout floor (10s), use 10s+RTT as the
minimum (increased on resends as before, of course).
* Always prod the clock update listeners, even if just to tell them that
the time hasn't changed much.
* Added support for explicit peer selection for individual tunnel pools,
which will be useful in debugging but not recommended for use by normal
end users.
* More aggressively search for the next hop's routerInfo on tunnel join.
* Give messages received via inbound tunnels that are bound to remote
locations sufficient time (taking into account clock skew).
* Give alternate direct send messages sufficient time (10s min, not 5s)
* Always give the end to end data message the explicit timeout (though the
old default was sufficient before)
* No need to give end to end messages an insane expiration (+2m), as we
are already handling skew on the receiving side.
* Don't complain too loudly about expired TunnelCreateMessages (at least,
not until after all those 0.5 and 0.5.0.1 users upgrade ;)
* Properly keep the sendBps stat
* When running the router with router.keepHistory=true, log more data to
messageHistory.txt
* Logging updates
* Minor formatting updates
* Filter HTTP response headers in the eepproxy, forcing Connection: close
so that broken (/malicious) webservers can't allow persistent
connections. All HTTP compliant browsers should now always close the
socket.
* Enabled the GZIPInputStream's cache (they were'nt cached before)
* Make sure our first send is always a SYN (duh)
* Workaround for some buggy compilers
* Loop while starting up the I2PTunnel instances, in case the I2CP
listener isn't up yet (thanks detonate!)
* Implement custom reusable GZIP streams to both reduce memory churn
and prevent the exposure of data in the standard GZIP header (creation
time, OS, etc). This is RFC1952 compliant, and backwards compatible,
though has only been tested within the confines of I2P's compression use
(DataHelper.[de]compress).
* Preemptively support the next protocol version, so that after the 0.5.0.2
release, we'll be able to drop protocol=2 to get rid of 0.5 users.
* Fix one substantial OOM cause (session tag manager was only dropping
tags once the critical limit was met, rather than honoring their
expiration) (duh)
* Lots of small memory fixes
* Double the allowable concurrent outstanding tunnel build tasks (20)
* Really disable the streaming lib packet caching
* Synchronized a message handling point in the SDK (even though its use is
already essentially single threaded, its better to play it safe)
* Don't add new RepublishLeaseSetJobs on failure, just requeue up the
existing one (duh)
* Throttle the number of concurrent pending tunnel builds across all
pools, in addition to simply throttling the number of new requests per
minute for each pool individually. This should avoid the cascading
failure when tunnel builds take too long, as no new builds will be
created until the previous ones are handled.
* Factored out and extended the DataHelper's unit tests for dealing with
long and date formatting.
* Explicitly specify the HTTP auth realm as "i2prouter", though this
alone doesn't address the bug where jetty asks for authentication too
much. (thanks orion!)
* Updated the StreamSinkServer to ignore all read bytes, rather than write
them to the filesystem.
* Don't rerequest leaseSets if there are already pending requests
* Reverted the insufficiently tested caching in the DSA/SHA1 impl, and
temporary disabled the streaming lib packet caching.
* Reduced the resend RTT penalty to 10s
* Further streaming lib caching improvements
* Reduce the minimum RTT (used to calculate retry timeouts), but also
increase the RTT on resends.
* Lower the default message size to 4KB from 16KB to further reduce the
chance of failed fragmentation.
* Extend tunnel rebuild throttling to include fallback rebuilds
* If there are less than 20 routers known, don't drop the last 20 (to help
avoid dropping all peers under catastrophic failures)
* New stats for end to end messages - "client.leaseSetFoundLocally",
"client.leaseSetFoundRemoteTime", and "client.leaseSetFailedRemoteTime"
* Cache temporary memory allocation in the DSA's SHA1 impl, and the packet
data in the streaming lib.
* Fixed a streaming lib bug where the connection initiator would fail the
stream if the ACK to their SYN was lost.
* Temporary workaround for the I2CP disconnect bug (have the streaminglib
try to automatically reconnect on accept()/connect(..)).
* Loop check for expired lease republishing (just in case)
* Adjusted (and fixed...) the timestamper change detection
* Deal with a rare reordering bug at the beginning of a stream (so we
don't drop it unnecessarily)
* Cleaned up some dropped message handling in the router
* Reduced job queue churn when dealing with a large number of tunnels by
sharing an expiration job
* Keep a separate list of the most recent CRIT messages (shown on the
logs.jsp). This way they don't get buried among any other messages.
* For clarity, display the tunnel variance config as "Randomization" on
the web console.
* If lease republishing fails (boo! hiss!) try it again
* Actually fix the negative jobLag in the right place (this time)
* Allow reseeding when there are less than 10 known peer references
* Lots of logging updates.
* Allow the streaming lib resend frequency to drop down to 20s as the
minimum, so that up to 2 retries can get sent on an http request.
* Add further limits to failsafe tunnels.
* Keep exploratory and client tunnel testing and building stats separate.
* Only use the 60s period for throttling tunnel requests due to transient
network overload.
* Rebuild tunnels earlier (1-3m before expiration, by default)
* Cache the next hop's routerInfo for participating tunnels so that the
tunnel participation doesn't depend on the netDb.
* Fixed a long standing bug in the streaming lib where we wouldn't always
unchoke messages when the window size grows.
* Make sure the window size never reaches 0 (duh)
* Only build new extra tunnels on failure if we don't have enough
* Fix a fencepost in the tunnel building so that e.g. a variance of
2 means +/- 2, not +/- 1 (thanks dm!)
* Avoid an NPE on client disconnect
* Never select a shitlisted peer to participate in a tunnel
* Have netDb store messages timeout after 10s, not the full 60s (duh)
* Keep session tags around for a little longer, just in case (grr)
* Cleaned up some closing event issues on the streaming lib
* Stop bundling the jetty 5.1.2 and updated wrapper.config in the update
so that 0.4.* users will need to do a clean install, but we don't need
to shove an additional 2MB in each update to those already on 0.5.
* Imported the susimail css (oops, thanks susi!)
* (Merged the 0.5-pre branch back into CVS HEAD)
* Replaced the old tunnel routing crypto with the one specified in
router/doc/tunnel-alt.html, including updates to the web console to view
and tweak it.
* Provide the means for routers to reject tunnel requests with a wider
range of responses:
probabalistic rejection, due to approaching overload
transient rejection, due to temporary overload
bandwidth rejection, due to persistent bandwidth overload
critical rejection, due to general router fault (or imminent shutdown)
The different responses are factored into the profiles accordingly.
* Replaced the old I2CP tunnel related options (tunnels.depthInbound, etc)
with a series of new properties, relevent to the new tunnel routing code:
inbound.nickname (used on the console)
inbound.quantity (# of tunnels to use in any leaseSets)
inbound.backupQuantity (# of tunnels to keep in the ready)
inbound.length (# of remote peers in the tunnel)
inbound.lengthVariance (if > 0, permute the length by adding a random #
up to the variance. if < 0, permute the length
by adding or subtracting a random # up to the
variance)
outbound.* (same as the inbound, except for the, uh, outbound tunnels
in that client's pool)
There are other options, and more will be added later, but the above are
the most relevent ones.
* Replaced Jetty 4.2.21 with Jetty 5.1.2
* Compress all profile data on disk.
* Adjust the reseeding functionality to work even when the JVM's http proxy
is set.
* Enable a poor-man's interactive-flow in the streaming lib by choking the
max window size.
* Reduced the default streaming lib max message size to 16KB (though still
configurable by the user), also doubling the default maximum window
size.
* Replaced the RouterIdentity in a Lease with its SHA256 hash.
* Reduced the overall I2NP message checksum from a full 32 byte SHA256 to
the first byte of the SHA256.
* Added a new "netId" flag to let routers drop references to other routers
who we won't be able to talk to.
* Extended the timestamper to get a second (or third) opinion whenever it
wants to actually adjust the clock offset.
* Replaced that kludge of a timestamp I2NP message with a full blown
DateMessage.
* Substantial memory optimizations within the router and the SDK to reduce
GC churn. Client apps and the streaming libs have not been tuned,
however.
* More bugfixes thank you can shake a stick at.
2005-02-13 jrandom
* Updated jbigi source to handle 64bit CPUs. The bundled jbigi.jar still
only contains 32bit versions, so build your own, placing libjbigi.so in
your install dir if necessary. (thanks mule!)
* Added support for libjbigi-$os-athlon64 to NativeBigInteger and CPUID
(thanks spaetz!)
* Added meaningful support for adjusting the preferred message size in the
streaming lib by setting the i2p.streaming.maxMessageSize=32768 (or
whatever). The other side will mimic a reduction (but never an increase).
* Always make sure to use distinct ConnectionOption objects for each
connection (duh)
* Reduced the default ACK delay to 500ms on in the streaming lib
* Only shrink the streaming window once per window
* Don't bundle a new jetty.xml with updates
* Catch another local routerInfo corruption issue on startup.
* Caught a series of (previously unhandled) errors caused by requeueing
messages that had timed out on the TCP transport (thanks mae^!)
* Reduce the barrier to dropping session tags on streaming lib resends -
every fourth send should drop the tags, forcing ElGamal encryption. This
will help speed up the recovery after a disconnect, rather than the drop
every fifth send.
* Handle unexpected network read errors more carefully (thanks parg!)
* Added more methods to partially compare (DataHelper) and display
arrays (Base64.encode).
* Exposed the AES encryptBlock/decryptBlock on the context.aes()
* Be more generous on the throttle when just starting up the router
* Fix a missing scheduled event in the streaming lib (caused after reset)
* Add a new DisconnectListener on the I2PSocketManager to allow
notification of session destruction.
* Make sure our own router identity is valid, and if it isn't, build a new
one and restart the router. Alternately, you can run the Router with
the single command line argument "rebuild" and it will do the same.
* Cleaned up the resending and choking algorithm in the streaming lib.
* Removed the read timeout override for I2PTunnel's httpclient, allowing
it to use the default for the streaming lib.
* Revised ack triggers in the streaming lib.
* Logging.
* Cleaned up the postinstall/startup scripts a bit more to handle winME,
and added windows info to the headless docs. (thanks ardvark!)
* Fixed a harmless (yet NPE inspiring) race during the final shutdown of
a stream (thanks frosk!)
* Add a pair of new stats for monitoring tunnel participation -
tunnel.participatingBytesProcessed (total # bytes transferred) and
tunnel.participatingBytesProcessedActive (total # bytes transferred for
tunnels whose byte count exceed the 10m average). This should help
further monitor congestion issues.
* Made the NamingService factory property public (thanks susi!)
* Catch another oddball case for a reset connection in the streaming lib.
* Add a dumpprofile.jsp page, called with ?peer=base64OfPeerHash, which
dumps the current state of that peer's profile. Instead of the full
base64, you can pass in however many characters you have and it will
return the first match found.
* Handle hard disconnects more gracefully within the streaming lib, and
log unmonitored events more aggressively.
* If we drop a peer after connection due to clock skew, log it to the
/logs.jsp#connectionlogs with relevent info. In addition, toss it in
the stat 'tcp.disconnectAfterSkew'.
* Fixed the formatting in the skew display
* Added an ERROR message that is fired once after we run out of
routerInfo files (thanks susi!)
* Set the connect timeout equal to the streaming lib's disconnect timeout
if not already specified (the I2PTunnel httpclient already enforces a
60s connect timeout)
* Fix for another connection startup problem in the streaming lib.
* Fix for a stupid error in the probabalistic drop (rand <= P, not > P)
* Adjust the capacity calculations so that tunnel failures alone in the
last 10m will not trigger a 0 capacity rank.
* Reenable the probabalistic drop on the TCP queues to deal with good old
fashioned bandwidth limiting. However, by default the probability is
rigged to reserve 0% of the queue free - meaning we just aggressively
fail messages in the queue if we're transferring too slowly. That
reservation factor can be increased with 'tcp.queueFreeFactor=0.25'
(or whatever) and the drop code can be disabled with the parameter
'tcp.dropProbabalistically=false'.
* Still penalize a peer on tunnel failure, but don't immediately drop
their capacity to 0.
* More aggressively ACK duplicates
* Randomize the timestamper period
* Display the clock skew on the connection logs when a peer sends it.
* Allow the timestamper to fix skews of up to 10 minutes
* Logging
* Fix the missing HTTP timeout, which was caused by the deferred syn used
by default. This, in turn, meant the I2PSocket creation doesn't fail
on .connect, but is unable to transfer any data in any direction. We now
detect that condition for the I2PTunnelHTTPClient and throw up the right
error page.
* Logging
* Use a simpler and less memory intensive job for processing outbound
client messages when the session is in mode=bestEffort. We can
immediately discard the data as soon as its sent the first time,
rather than wait for an ack, since we will never internally resend.
* Reduce some synchronization to avoid a rare deadlock
* Replaced 'localhost' with 127.0.0.1 in the i2ptunnel config, and special
case it within the tunnel controller.
* Script cleanup for building jbigi/jcpuid
* Logging
* Revised the buffering when reading from the SAM client and writing
to the stream. Also added a thread (sigh) so we don't block the
SAM client from giving us more messages for abnormally long periods
of time.
* Display the router version in the logs on startup (oft requested)
* Fix a race during the closing of a messageOutputStream
* Don't do a 'passive flush' while there are already outbound messages
unacked.
* Show the reseed link if up to 10 peers profiles are active (thanks
dburton!)
* Don't do a 'passive flush' while there are already outbound messages
unacked.
* Show the reseed link if up to 10 peers profiles are active (thanks
dburton!)
* Fix the recently broken SAM bridge (duh)
* Add a new pair of SAM apps - net.i2p.sam.client.SAMStreamSink and
net.i2p.sam.client.SAMStreamSend, mirroring the streaming lib's
StreamSink and StreamSend apps for transferring files.
* Make the passive flush timer fire more frequently.
* Toss in a small pool of threads (3) to execute the events queued up with
the SimpleTimer, as we do currently see the occational event
notification spiking up to a second or so.
* Implement a SAM client API in java, useful for event based streaming (or
for testing the SAM bridge)
* Added support to shut down the SAM bridge on OOM (useful if the SAM
bridge is being run outside of the router).
* Include the SAM test code in the sam.jar
* Remove an irrelevent warning message from SAM, which was caused by
perfectly normal operation due to a session being closed.
* Removed some unnecessary synchronization in the streaming lib's
PacketQueue
* More quickly clean up the memory used by the streaming lib by
immediately killing each packet's resend job as soon as it is ACKed (or
cancelled), so that there are no longer any valid pointers to the
(potentially 32KB) packet.
* Fixed the timestamps dumped to stdout when debugging the PacketHandler.
* Drop packets that would expand our inbound window beyond our maximum
buffer size (default 32 messages)
* Always read the ACK/NACK data from the verified packets received, even
if we are going to drop them
* Always adjust the window when there are messages ACKed, though do not
change its size except as before.
* Streamlined some synchronization in the router's I2CP handling
* Streamlined some memory allocation in the SAM bridge
* Default the streaming lib to disconnect on inactivity, rather than send
an empty message.
this still doesnt get the BT to where it needs to be, or fix the timeout problem,
but i dont like having so many commits outstanding and these updates are sound
2004-12-01 jrandom
* Strip out any of the Accept-* HTTP header lines, and always make sure to
include the forged User-agent header.
* Adjust the default read timeout on the eepproxy to 60s, unless
overridden.
* Minor tweak on stream shutdown.
* Render the burst rate fields on /config.jsp properly (thanks ugha!)
* Build in a simple timeout to flush data queued into the I2PSocket but
not yet flushed.
* Don't explicitly flush after each SAM stream write, but leave it up to
the [nonblocking] passive flush.
* Don't whine about 10-99 connection events occurring in a second
* Don't wait for completion of packets that will not be ACKed (duh)
* Adjust the congestion window, even if the packet was resent (duh)
* Make sure to wake up any blocking read()'s when the MessageInputStream
is close()ed (duh)
* Never wait more than the disconnect timeout for a write to complete
* Reduced contention for local client delivery
* Drop the new code that munges the wrapper.config. Instead, updates that
need to change it will include their own wrapper.config in the
i2pupdate.zip, overwriting the existing file. If the file
"wrapper.config.updated" is included, it is deleted at first opportunity
and the router shut down, displaying a notice that the router must be
started again cleanly to allow the changes to the wrapper.config to take
effect.
* Properly stop accept()ing I2PSocket connections if we close down the
session (duh).
* Make sure we cancel any outstanding Packets in flight when a connection
is terminated (thanks susi!)
* Split up the I2PTunnel closing a little further.
* Some cleanup and bugfixes for the IP address detection code where we
only consider connections that have actually sent and received messages
recently as active, rather than the mere presence of a TCP socket as
activity.
* Fix for a fast loop caused by a race in the new streaming library (thanks
DrWoo, frontier, pwk_, and thetower!)
* Minor updates to the SimpleTimer and Connection to help track down a
high CPU usage problem (dumping debug info to stdout/wrapper.log if too
many events/tasks fire in a second)
* Minor fixes for races on client disconnects (causing NPEs)
if you want to get this data and you're running outside the router, just toss on a
"-Dstat.logFilters=* -Dstat.logFile=proxy.stats" to the java command line. the resulting
file can be parsed w/ java -cp lib/i2p.jar net.i2p.stat.StatLogSplitter proxy.stats, then fed
into gnuplot or whatever
further and cancel all of the tags we're using for that peer so that we can react to their
potential restart / tag loss quicker.
* use the minimum resend delay as the base to be exponentiated if our RTT is too low
(so we resend less)
* dont be such a wuss when flushing a closed stream
