YXORP
|
|
YXORP is a reverse proxy for the HTTP protocol.
The main focus for YXORP is checking the validity of the traffic, i.e. what would be called 'application layer firewalling' or 'secure reverse proxy'
in commercial products. It can check, validate, and change just about anything about a HTTP request or response, can load balance, and all other normal reversed proxy functions.
YXORP aims to conform to RFC 2616, RFC 2518, RFC 2109 and other applicable standards.
What YXORP does is receive requests, break all fields and headers apart and feed the information in them to a script engine. The scripts can change
all fields except for a few that are required for the connections to work correctly; for instance the Connection: header and the Content-Length header. After the scripts have finished, YXORP then sends the request - including any
changes that have been made - on to the web server.
YXORP is written in plain C; it should run on any Unix variant as long as it supports reasonably recent standards. It requires
a couple of libraries to build, like libxml2, openssl, and pcre - all mainstream Unix versions have these by default or allow the
required development libraries to be easily installed.
There used to be two sets of releases - development and stable.
Since the amount of development is rather low in recent years
- because YXORP is a mature and stable software -
I've decided to no longer differentiate between these. Especially
since most major sites that run YXORP that I'm aware of use
the 'development' releases anyway.
In case of bugs, the last two releases will be supported; bug
fixes may still be produced for older versions but this will be
decided on a case by case basis.
Changelog for release 2.34 (latest):
-
fix for #3586630, not processing the Connection:
header correctly in the presence of token values
other than close.
-
fix for issue when serving chunked, then unchunked
content from a cache - the content-length was not
correctly set.
Changelog for release 2.33:
- changed processing of basicauth with regard to
ldap. Previously, using basic_auth_check would
cause yxorp to read the clear text password from
the ldap. Now, it uses the credentials in the
basic authentication to try a simple bind to the
ldap. New attribute to support this is
bindfmt on the ldapsource tag.
-
basicauth credentials verified with an ldap server
are now cached. The time that credentials are
cached are set by the cachetime attribute on
the ldapsource tag. The table is periodically
cleaned up; this is governed by the globalconfig
setting basicauthmaintenanceinterval.
- added Via: header processing, including new
globalconfig settings viareportversionstringtoclient,
viareportpackagenametoclient, viareporthosttoclient,
viareportversionstringtoserver, viareportpackagenametoserver,
viareporthosttoserver. Default is to include a full
entry into an existing Via: header, or create a new
one for both requests and responses.
- fixed bugs dealing with setting the xlateid attribute
in customized header definitions; a) the default
setting for xlateid was incorrect; b) configured
setting of xlateid was not always completely copied
into the actual configuration.
- issue warning when configure has not detected a
usable ldap library
- include ldap support in showbuild and daemon output
- changed default for options method to allow
outbound entities. The options method is still
disabled by default [thanks to Rene Klootwijk]
- removed all ldap code related to ldapsource
and ldapserver tags
- started new ldap implementation based on rule
functions; functions added are ldap_set_serveruri,
ldap_set_searchdn, ldap_set_searchpw, ldap_set_searchscope,
ldap_search, ldap_set_searchbase, ldap_init, ldap_bind.
Also, changed the interface for checking basic
authentication; old function basic_auth_check was
removed; functions added are basicauth_add,
basicauth_check, basicauth_getuser, basicauth_getpass,
basicauth_reject for processing basic authentication.
Effectively, with this new interface, the 'old'
technology of setting basic authentication strings
in the configuration is still available, and
is now augmented by a very flexible interface to
ldap; the ldap query results can be cached in the
basic authentication table for optimization.
- fix for #3024257, new globalconfig items
maxchunkheaderlength and maxchunkheaderreadretries.
- fix for bug in the rewrite of location: headers
which could cause malformed headers and/or
duplicate headers [thanks to Rene Klootwijk for
identifying this issue]
Changelog for release 2.32:
- client states can now be mapped by including
function calls in a rule that derive some
information from the request that is unique
enough to reliably map into a state entry. An
example of such information might be a jsession
cookie that an application server has generated,
or some unique bits in the URL. The new functions
are setclientstateid() and setclientstateidgenerate().
- entries where the URI has parameters are no
longer added to the cache.
- the generated configuration, ie. from yxorpconfig -r,
now includes a read-only attribute that shows whether
or not a listener has opened the socket it is associated
with. The attribute is named open.
- added a new section of information-only output
to generated configuration ie. yxorpconfig -r. The
tag name of this section is daemon.
- added comment tags to the various sections in the
generated configuration.
- mkinstallconfig script now also installs the dtd.
- header Authentication-Info: added to the default
table.
- header Cookie2: added to the default header table.
- build problem in ypriv fixed [thanks to Ferenc].
- mode attribute in virtualserver added to dtd.
- adding support for sni, rfc4366/3546. New tag to
be included inside listener tag is certlist, takes
attributes of hostname, certfile, certpasswd,
cafile.
- certificates defined on a listener (in sni context,
these are the default certificates that a non-sni
capable browser will get) can now be updated.
- changed default header buffer check for Server:
header not to reject, because Apache2's default
may send out a Server: header that violates RFC2616.
- many changes to the sources to build without
warnings, in spite of a higher default warning
level.
- added globalconfiguration item localhostname, this
allows to set which name will be reported in for
instance Via: and X-Cache: headers.
- added X-Cache: header to header table.
- added globalconfiguration item generatexcache to
control whether X-Cache headers are generated. Note
that X-Cache headers are only generated if caching
is applicable to the request; if the request is not
cachable, no header is generated irrespective of
the setting of generatexcache.
- added weighted random scheduling, virtualserver
attribute schedule value "weightedrandom".
Also, weight tag on virtualserver real tag.
- removed mode parameter from virtualserver,
replaced by clearrealservers tag.
- added add and remove tags to virtualserver
real tag.
- added virtualserver command.
- several changes to fix a limitation in the maximum
content length. Previously, this was limited at
2Gb; now, it is unlimited.
- fixed a memory leak occurring when server side
ssl was used.
- added globalconfiguration item workerthreadstacksize
to enable tuning of memory mapping in case yxorp
is running many threads. In this scenario, most
platform defaults for the thread stack size are
much larger than yxorp requires, and issues with
system-imposed limits may arise. Preferably, though,
platform tools (like ulimit) should be used; this
configuration item exists only where using
the platform tool is not practicable.
- the default error log fmt no longer contains
%clientiplookup.
- feature request 1860197:
attribute added on listener tag, pvtkeyfile,
enables to load private key from a separate
file instead of the same file as the certfile.
If this attribute is not set, yxorp will try
to load the private key from the certfile.
- feature request 1860197:
attribute added on certlist tag, pvtkeyfile,
enables to load private key from a separate
file instead of the same file as the certfile.
If this attribute is not set, yxorp will try
to load the private key from the certfile.
- ssl items in a listener could be initialized
twice [thanks to Rene Klootwijk].
- fixed bug in message formatting when running
on a system without functioning name resolver
[thanks to Rene Klootwijk].
- added checks if sufficient entropy can be found
for use by OpenSSL. If not, yxorp will refuse
to start [thanks to Rene Klootwijk].
- feature request 1860197:
yxorp no longer checks where the configuration
files reside (including ssl certs, keys, etc).
- Please note when upgrading from previous versions: feature request 1860195:
yxorp by default now installs binaries for
both the daemon and tools in /.../sbin. When upgrading, please remove the old version before running 'make install', and check your init scripts for the correct path.
- added new function strremove to rule language.
- added clientprotocol and serverprotocol as
formats for request and error logs. Default
for both logs is now serverprotocol.
- changed processing of wakeup for out-of-service
real servers. If all defined servers are out
of service, always try to wake one up.
- Location: headers that contain a hostname
(in absoluteURI form) are now rewritten to
the hostname that the client originally
used. This can be disabled by the new
setting rewritelocationheader in globalconfig.
- changed default check for content-type header
to text.
- added option renameonrotate in globalconfiguration
if 1, yxorp will append a time stamp to the
old (rotated) log file after closing it. If 0,
which is default, yxorp will just close and
then reopen the log files.
- added functions setsslserverconnecttimeout,
setsslserverreadtimeout, setsslserverwritetimeout,
setsslserverclosetimeout, setconnectservertimeout,
setreadfromservertimeout setwritetoservertimeout;
these can be used to modify the timeout values
used for the server connection for specific
requests.
Changelog for release 2.31:
- make sure bytes in threadlog-hexdump are not
sign extended. Same for debugging hexdump.
- retry connecting to server if the initial
read from the server returns 0 ie. session
shutdown.
- yxorpconfig -rv now only shows enabled headers.
Same for methods.
- report the serverconnectionretrycounter in
rdlog internals output.
- added folding attribute into header table. At
this time, it will not yet be enabled for any
headers besides XFF.
- added apparently forgotten locking in
count_yxorp_threads()
- changed defaults for methods OPTIONS, PUT, DELETE;
these are now by default enabled if the
methodgroups=rfc2616 attribute is enabled in the
globalconfiguration.
- changed default for header DAV buffercheck, is now
rfc2616_text.
- changed processing of inbound entities without
Content-Length: header. If the method carries the
dav or msext groups, yxorp will assume that no
inbound entity is present if no Content-Length:
header is present on the request.
- added function killclientstate(), this will
immediately remove the entire client state
associated with the current request. Synonym
is clearclientstate().
- added function killsticky(), synonym clearsticky()
which clears the sticky mapping for the current
request.
- added function setclientstatefastage() to set a
faster age timer on an individual client state
entry. This is to be used by a rule in the situation
that a session is ending, but some requests may
still be coming in that we will want to handle on
the original state.
- changed the behaviour of the stickyloss and sorry rules.
These rule types no longer run as reject rules and
implicitly end the request. Sorry rules may now be
used to cause a retry to another server (including
scheduling to another virtualserver), and or with changed
request fields. Special variables connectionretries
and maxconnectionretries are set in the sorryrule
context to allow sorryrules to check whether retries
are exhausted. Stickyloss rules may either cause
a redirect, or an explicit reject. It is also possible
to forward the request to another server group, even
though this probably will not make sense, since
no session context will be available on this other
server. In the case of a stickylost rule,
the special variable stickylost is set to a value of 1.
Both for the stickylost and the sorry rules, using a
reject in a rule will cause the defined reject rule
to run before the end of the request processing.
- added option -l to yxorp command to load configuration
from the file that was set on the command that originally
started the daemon.
- Also, option to the yxorp command -l can be set as
--load, and -V as --version.
- Several options (-d, -D, -v, -X) now cause an error if
yxorp was built without debugging.
- Added code to more predictably detect if the session
from yxorp to a backend server is available for reuse.
- Increased default maximum for the thread pool to 200;
increased the minimum free to 6; increased initial to
20. Since 2.26 threads exit after a configurable time
of inactivity, so the increased numbers will settle at
minimum amounts for an idle yxorp after some time, while
this may prevent problems for heavy users. Also added a
new keyword startatoverrun that governs how many threads
are started if minfree is reached.
- done some updates to make parsing and tools code more
maintainable.