Imported from SVN by Bitbucket

This commit is contained in:
2015-03-31 20:26:20 +00:00
committed by bitbucket
commit ceb7984dec
212 changed files with 49537 additions and 0 deletions

2
MANIFEST.in Executable file
View File

@@ -0,0 +1,2 @@
recursive-include caotek *
global-exclude *pyc

View File

@@ -0,0 +1,119 @@
Metadata-Version: 1.0
Name: Paste
Version: 1.7.5.1
Summary: Tools for using a Web Server Gateway Interface stack
Home-page: http://pythonpaste.org
Author: Ian Bicking
Author-email: ianb@colorstudy.com
License: MIT
Description: These provide several pieces of "middleware" (or filters) that can be nested to build web applications. Each
piece of middleware uses the WSGI (`PEP 333`_) interface, and should
be compatible with other middleware based on those interfaces.
.. _PEP 333: http://www.python.org/peps/pep-0333.html
Includes these features...
Testing
-------
* A fixture for testing WSGI applications conveniently and in-process,
in ``paste.fixture``
* A fixture for testing command-line applications, also in
``paste.fixture``
* Check components for WSGI-compliance in ``paste.lint``
Dispatching
-----------
* Chain and cascade WSGI applications (returning the first non-error
response) in ``paste.cascade``
* Dispatch to several WSGI applications based on URL prefixes, in
``paste.urlmap``
* Allow applications to make subrequests and forward requests
internally, in ``paste.recursive``
Web Application
---------------
* Run CGI programs as WSGI applications in ``paste.cgiapp``
* Traverse files and load WSGI applications from ``.py`` files (or
static files), in ``paste.urlparser``
* Serve static directories of files, also in ``paste.urlparser``; also
in that module serving from Egg resources using ``pkg_resources``.
Tools
-----
* Catch HTTP-related exceptions (e.g., ``HTTPNotFound``) and turn them
into proper responses in ``paste.httpexceptions``
* Several authentication techniques, including HTTP (Basic and
Digest), signed cookies, and CAS single-signon, in the
``paste.auth`` package.
* Create sessions in ``paste.session`` and ``paste.flup_session``
* Gzip responses in ``paste.gzip``
* A wide variety of routines for manipulating WSGI requests and
producing responses, in ``paste.request``, ``paste.response`` and
``paste.wsgilib``
Debugging Filters
-----------------
* Catch (optionally email) errors with extended tracebacks (using
Zope/ZPT conventions) in ``paste.exceptions``
* Catch errors presenting a `cgitb
<http://python.org/doc/current/lib/module-cgitb.html>`_-based
output, in ``paste.cgitb_catcher``.
* Profile each request and append profiling information to the HTML,
in ``paste.debug.profile``
* Capture ``print`` output and present it in the browser for
debugging, in ``paste.debug.prints``
* Validate all HTML output from applications using the `WDG Validator
<http://www.htmlhelp.com/tools/validator/>`_, appending any errors
or warnings to the page, in ``paste.debug.wdg_validator``
Other Tools
-----------
* A file monitor to allow restarting the server when files have been
updated (for automatic restarting when editing code) in
``paste.reloader``
* A class for generating and traversing URLs, and creating associated
HTML code, in ``paste.url``
The latest version is available in a `Subversion repository
<http://svn.pythonpaste.org/Paste/trunk#egg=Paste-dev>`_.
For the latest changes see the `news file
<http://pythonpaste.org/news.html>`_.
Keywords: web application server wsgi
Platform: UNKNOWN
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python
Classifier: Topic :: Internet :: WWW/HTTP
Classifier: Topic :: Internet :: WWW/HTTP :: Dynamic Content
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: Internet :: WWW/HTTP :: WSGI
Classifier: Topic :: Internet :: WWW/HTTP :: WSGI :: Application
Classifier: Topic :: Internet :: WWW/HTTP :: WSGI :: Middleware
Classifier: Topic :: Internet :: WWW/HTTP :: WSGI :: Server
Classifier: Framework :: Paste

View File

@@ -0,0 +1,286 @@
MANIFEST.in
setup.cfg
setup.py
Paste.egg-info/PKG-INFO
Paste.egg-info/SOURCES.txt
Paste.egg-info/dependency_links.txt
Paste.egg-info/entry_points.txt
Paste.egg-info/namespace_packages.txt
Paste.egg-info/not-zip-safe
Paste.egg-info/requires.txt
Paste.egg-info/top_level.txt
docs/DeveloperGuidelines.txt
docs/StyleGuide.txt
docs/conf.py
docs/developer-features.txt
docs/do-it-yourself-framework.txt
docs/future.txt
docs/index.txt
docs/license.txt
docs/news.txt
docs/paste-httpserver-threadpool.txt
docs/testing-applications.txt
docs/url-parsing-with-wsgi.txt
docs/_build/DeveloperGuidelines.html
docs/_build/StyleGuide.html
docs/_build/developer-features.html
docs/_build/do-it-yourself-framework.html
docs/_build/future.html
docs/_build/genindex.html
docs/_build/index.html
docs/_build/license.html
docs/_build/modindex.html
docs/_build/news.html
docs/_build/paste-httpserver-threadpool.html
docs/_build/py-modindex.html
docs/_build/search.html
docs/_build/testing-applications.html
docs/_build/url-parsing-with-wsgi.html
docs/_build/community/index.html
docs/_build/community/mailing-list.html
docs/_build/community/repository.html
docs/_build/download/index.html
docs/_build/include/contact.html
docs/_build/include/reference_header.html
docs/_build/modules/auth.auth_tkt.html
docs/_build/modules/auth.basic.html
docs/_build/modules/auth.cas.html
docs/_build/modules/auth.cookie.html
docs/_build/modules/auth.digest.html
docs/_build/modules/auth.form.html
docs/_build/modules/auth.grantip.html
docs/_build/modules/auth.multi.html
docs/_build/modules/auth.open_id.html
docs/_build/modules/cascade.html
docs/_build/modules/cgiapp.html
docs/_build/modules/cgitb_catcher.html
docs/_build/modules/debug.debugapp.html
docs/_build/modules/debug.fsdiff.html
docs/_build/modules/debug.prints.html
docs/_build/modules/debug.profile.html
docs/_build/modules/debug.watchthreads.html
docs/_build/modules/debug.wdg_validate.html
docs/_build/modules/errordocument.html
docs/_build/modules/evalexception.html
docs/_build/modules/exceptions.html
docs/_build/modules/fileapp.html
docs/_build/modules/fixture.html
docs/_build/modules/gzipper.html
docs/_build/modules/httpexceptions.html
docs/_build/modules/httpheaders.html
docs/_build/modules/httpserver.html
docs/_build/modules/lint.html
docs/_build/modules/pony.html
docs/_build/modules/progress.html
docs/_build/modules/proxy.html
docs/_build/modules/recursive.html
docs/_build/modules/registry.html
docs/_build/modules/reloader.html
docs/_build/modules/request.html
docs/_build/modules/response.html
docs/_build/modules/session.html
docs/_build/modules/transaction.html
docs/_build/modules/translogger.html
docs/_build/modules/url.html
docs/_build/modules/urlmap.html
docs/_build/modules/urlparser.html
docs/_build/modules/util.import_string.html
docs/_build/modules/util.multidict.html
docs/_build/modules/wsgilib.html
docs/_build/modules/wsgiwrappers.html
docs/community/index.txt
docs/community/mailing-list.txt
docs/community/repository.txt
docs/download/index.txt
docs/include/contact.txt
docs/include/reference_header.txt
docs/modules/auth.auth_tkt.txt
docs/modules/auth.basic.txt
docs/modules/auth.cas.txt
docs/modules/auth.cookie.txt
docs/modules/auth.digest.txt
docs/modules/auth.form.txt
docs/modules/auth.grantip.txt
docs/modules/auth.multi.txt
docs/modules/cascade.txt
docs/modules/cgiapp.txt
docs/modules/cgitb_catcher.txt
docs/modules/debug.debugapp.txt
docs/modules/debug.fsdiff.txt
docs/modules/debug.prints.txt
docs/modules/debug.profile.txt
docs/modules/debug.watchthreads.txt
docs/modules/debug.wdg_validate.txt
docs/modules/errordocument.txt
docs/modules/evalexception.txt
docs/modules/exceptions.txt
docs/modules/fileapp.txt
docs/modules/fixture.txt
docs/modules/gzipper.txt
docs/modules/httpexceptions.txt
docs/modules/httpheaders.txt
docs/modules/httpserver.txt
docs/modules/lint.txt
docs/modules/pony.txt
docs/modules/progress.txt
docs/modules/proxy.txt
docs/modules/recursive.txt
docs/modules/registry.txt
docs/modules/reloader.txt
docs/modules/request.txt
docs/modules/response.txt
docs/modules/session.txt
docs/modules/transaction.txt
docs/modules/translogger.txt
docs/modules/url.txt
docs/modules/urlmap.txt
docs/modules/urlparser.txt
docs/modules/util.import_string.txt
docs/modules/util.multidict.txt
docs/modules/wsgilib.txt
docs/modules/wsgiwrappers.txt
paste/__init__.py
paste/cascade.py
paste/cgiapp.py
paste/cgitb_catcher.py
paste/config.py
paste/errordocument.py
paste/fileapp.py
paste/fixture.py
paste/flup_session.py
paste/gzipper.py
paste/httpexceptions.py
paste/httpheaders.py
paste/httpserver.py
paste/lint.py
paste/modpython.py
paste/pony.py
paste/progress.py
paste/proxy.py
paste/recursive.py
paste/registry.py
paste/reloader.py
paste/request.py
paste/response.py
paste/session.py
paste/transaction.py
paste/translogger.py
paste/url.py
paste/urlmap.py
paste/urlparser.py
paste/wsgilib.py
paste/wsgiwrappers.py
paste/auth/__init__.py
paste/auth/auth_tkt.py
paste/auth/basic.py
paste/auth/cas.py
paste/auth/cookie.py
paste/auth/digest.py
paste/auth/form.py
paste/auth/grantip.py
paste/auth/multi.py
paste/auth/open_id.py
paste/cowbell/__init__.py
paste/debug/__init__.py
paste/debug/debugapp.py
paste/debug/doctest_webapp.py
paste/debug/fsdiff.py
paste/debug/prints.py
paste/debug/profile.py
paste/debug/testserver.py
paste/debug/watchthreads.py
paste/debug/wdg_validate.py
paste/evalexception/__init__.py
paste/evalexception/evalcontext.py
paste/evalexception/middleware.py
paste/evalexception/media/MochiKit.packed.js
paste/evalexception/media/debug.js
paste/evalexception/media/minus.jpg
paste/evalexception/media/plus.jpg
paste/exceptions/__init__.py
paste/exceptions/collector.py
paste/exceptions/errormiddleware.py
paste/exceptions/formatter.py
paste/exceptions/reporter.py
paste/exceptions/serial_number_generator.py
paste/util/PySourceColor.py
paste/util/UserDict24.py
paste/util/__init__.py
paste/util/classinit.py
paste/util/classinstance.py
paste/util/converters.py
paste/util/dateinterval.py
paste/util/datetimeutil.py
paste/util/doctest24.py
paste/util/filemixin.py
paste/util/finddata.py
paste/util/findpackage.py
paste/util/import_string.py
paste/util/intset.py
paste/util/ip4.py
paste/util/killthread.py
paste/util/looper.py
paste/util/mimeparse.py
paste/util/multidict.py
paste/util/quoting.py
paste/util/scgiserver.py
paste/util/string24.py
paste/util/subprocess24.py
paste/util/template.py
paste/util/threadedprint.py
paste/util/threadinglocal.py
tests/__init__.py
tests/test_cgiapp.py
tests/test_cgitb_catcher.py
tests/test_config.py
tests/test_doctests.py
tests/test_errordocument.py
tests/test_fileapp.py
tests/test_fixture.py
tests/test_grantip.py
tests/test_gzipper.py
tests/test_httpheaders.py
tests/test_import_string.py
tests/test_multidict.py
tests/test_profilemiddleware.py
tests/test_proxy.py
tests/test_recursive.py
tests/test_registry.py
tests/test_request.py
tests/test_request_form.py
tests/test_response.py
tests/test_session.py
tests/test_template.txt
tests/test_urlmap.py
tests/test_urlparser.py
tests/test_wsgiwrappers.py
tests/test_auth/__init__.py
tests/test_auth/test_auth_cookie.py
tests/test_auth/test_auth_digest.py
tests/test_exceptions/__init__.py
tests/test_exceptions/test_error_middleware.py
tests/test_exceptions/test_formatter.py
tests/test_exceptions/test_httpexceptions.py
tests/test_exceptions/test_reporter.py
tests/test_util/__init__.py
tests/test_util/test_datetimeutil.py
tests/test_util/test_mimeparse.py
tests/urlparser_data/__init__.py
tests/urlparser_data/secured.txt
tests/urlparser_data/deep/sub/Main.txt
tests/urlparser_data/find_file/index.txt
tests/urlparser_data/hook/__init__.py
tests/urlparser_data/hook/app.py
tests/urlparser_data/hook/index.py
tests/urlparser_data/not_found/__init__.py
tests/urlparser_data/not_found/recur/__init__.py
tests/urlparser_data/not_found/recur/isfound.txt
tests/urlparser_data/not_found/simple/__init__.py
tests/urlparser_data/not_found/simple/found.txt
tests/urlparser_data/not_found/user/__init__.py
tests/urlparser_data/not_found/user/list.py
tests/urlparser_data/python/__init__.py
tests/urlparser_data/python/simpleapp.py
tests/urlparser_data/python/stream.py
tests/urlparser_data/python/sub/__init__.py
tests/urlparser_data/python/sub/simpleapp.py

View File

@@ -0,0 +1 @@

View File

@@ -0,0 +1,47 @@
[paste.app_factory]
cgi = paste.cgiapp:make_cgi_application [subprocess]
static = paste.urlparser:make_static
pkg_resources = paste.urlparser:make_pkg_resources
urlparser = paste.urlparser:make_url_parser
proxy = paste.proxy:make_proxy
test = paste.debug.debugapp:make_test_app
test_slow = paste.debug.debugapp:make_slow_app
transparent_proxy = paste.proxy:make_transparent_proxy
watch_threads = paste.debug.watchthreads:make_watch_threads
[paste.composite_factory]
urlmap = paste.urlmap:urlmap_factory
cascade = paste.cascade:make_cascade
[paste.filter_app_factory]
error_catcher = paste.exceptions.errormiddleware:make_error_middleware
cgitb = paste.cgitb_catcher:make_cgitb_middleware
flup_session = paste.flup_session:make_session_middleware [Flup]
gzip = paste.gzipper:make_gzip_middleware
httpexceptions = paste.httpexceptions:make_middleware
lint = paste.lint:make_middleware
printdebug = paste.debug.prints:PrintDebugMiddleware
profile = paste.debug.profile:make_profile_middleware [hotshot]
recursive = paste.recursive:make_recursive_middleware
# This isn't good enough to deserve the name egg:Paste#session:
paste_session = paste.session:make_session_middleware
wdg_validate = paste.debug.wdg_validate:make_wdg_validate_middleware [subprocess]
evalerror = paste.evalexception.middleware:make_eval_exception
auth_tkt = paste.auth.auth_tkt:make_auth_tkt_middleware
auth_basic = paste.auth.basic:make_basic
auth_digest = paste.auth.digest:make_digest
auth_form = paste.auth.form:make_form
grantip = paste.auth.grantip:make_grantip
openid = paste.auth.open_id:make_open_id_middleware [openid]
pony = paste.pony:make_pony
cowbell = paste.cowbell:make_cowbell
errordocument = paste.errordocument:make_errordocument
auth_cookie = paste.auth.cookie:make_auth_cookie
translogger = paste.translogger:make_filter
config = paste.config:make_config_filter
registry = paste.registry:make_registry_manager
[paste.server_runner]
http = paste.httpserver:server_runner

View File

@@ -0,0 +1 @@
paste

View File

@@ -0,0 +1 @@

View File

@@ -0,0 +1,15 @@
[Flup]
flup
[openid]
python-openid
[Paste]
[hotshot]
[subprocess]

View File

@@ -0,0 +1 @@
paste

View File

@@ -0,0 +1,17 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
try:
import pkg_resources
pkg_resources.declare_namespace(__name__)
except ImportError:
# don't prevent use of paste if pkg_resources isn't installed
from pkgutil import extend_path
__path__ = extend_path(__path__, __name__)
try:
import modulefinder
except ImportError:
pass
else:
for p in __path__:
modulefinder.AddPackagePath(__name__, p)

View File

@@ -0,0 +1,9 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
"""
Package for authentication/identification of requests.
The objective of this package is to provide single-focused middleware
components that implement a particular specification. Integration of
the components into a usable system is up to a higher-level framework.
"""

View File

@@ -0,0 +1,396 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
##########################################################################
#
# Copyright (c) 2005 Imaginary Landscape LLC and Contributors.
#
# Permission is hereby granted, free of charge, to any person obtaining
# a copy of this software and associated documentation files (the
# "Software"), to deal in the Software without restriction, including
# without limitation the rights to use, copy, modify, merge, publish,
# distribute, sublicense, and/or sell copies of the Software, and to
# permit persons to whom the Software is furnished to do so, subject to
# the following conditions:
#
# The above copyright notice and this permission notice shall be
# included in all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
# LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
# OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
# WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
##########################################################################
"""
Implementation of cookie signing as done in `mod_auth_tkt
<http://www.openfusion.com.au/labs/mod_auth_tkt/>`_.
mod_auth_tkt is an Apache module that looks for these signed cookies
and sets ``REMOTE_USER``, ``REMOTE_USER_TOKENS`` (a comma-separated
list of groups) and ``REMOTE_USER_DATA`` (arbitrary string data).
This module is an alternative to the ``paste.auth.cookie`` module;
it's primary benefit is compatibility with mod_auth_tkt, which in turn
makes it possible to use the same authentication process with
non-Python code run under Apache.
"""
import time as time_mod
try:
from hashlib import md5
except ImportError:
from md5 import md5
import Cookie
from paste import request
from urllib import quote as url_quote
from urllib import unquote as url_unquote
class AuthTicket(object):
"""
This class represents an authentication token. You must pass in
the shared secret, the userid, and the IP address. Optionally you
can include tokens (a list of strings, representing role names),
'user_data', which is arbitrary data available for your own use in
later scripts. Lastly, you can override the cookie name and
timestamp.
Once you provide all the arguments, use .cookie_value() to
generate the appropriate authentication ticket. .cookie()
generates a Cookie object, the str() of which is the complete
cookie header to be sent.
CGI usage::
token = auth_tkt.AuthTick('sharedsecret', 'username',
os.environ['REMOTE_ADDR'], tokens=['admin'])
print 'Status: 200 OK'
print 'Content-type: text/html'
print token.cookie()
print
... redirect HTML ...
Webware usage::
token = auth_tkt.AuthTick('sharedsecret', 'username',
self.request().environ()['REMOTE_ADDR'], tokens=['admin'])
self.response().setCookie('auth_tkt', token.cookie_value())
Be careful not to do an HTTP redirect after login; use meta
refresh or Javascript -- some browsers have bugs where cookies
aren't saved when set on a redirect.
"""
def __init__(self, secret, userid, ip, tokens=(), user_data='',
time=None, cookie_name='auth_tkt',
secure=False):
self.secret = secret
self.userid = userid
self.ip = ip
self.tokens = ','.join(tokens)
self.user_data = user_data
if time is None:
self.time = time_mod.time()
else:
self.time = time
self.cookie_name = cookie_name
self.secure = secure
def digest(self):
return calculate_digest(
self.ip, self.time, self.secret, self.userid, self.tokens,
self.user_data)
def cookie_value(self):
v = '%s%08x%s!' % (self.digest(), int(self.time), url_quote(self.userid))
if self.tokens:
v += self.tokens + '!'
v += self.user_data
return v
def cookie(self):
c = Cookie.SimpleCookie()
c[self.cookie_name] = self.cookie_value().encode('base64').strip().replace('\n', '')
c[self.cookie_name]['path'] = '/'
if self.secure:
c[self.cookie_name]['secure'] = 'true'
return c
class BadTicket(Exception):
"""
Exception raised when a ticket can't be parsed. If we get
far enough to determine what the expected digest should have
been, expected is set. This should not be shown by default,
but can be useful for debugging.
"""
def __init__(self, msg, expected=None):
self.expected = expected
Exception.__init__(self, msg)
def parse_ticket(secret, ticket, ip):
"""
Parse the ticket, returning (timestamp, userid, tokens, user_data).
If the ticket cannot be parsed, ``BadTicket`` will be raised with
an explanation.
"""
ticket = ticket.strip('"')
digest = ticket[:32]
try:
timestamp = int(ticket[32:40], 16)
except ValueError, e:
raise BadTicket('Timestamp is not a hex integer: %s' % e)
try:
userid, data = ticket[40:].split('!', 1)
except ValueError:
raise BadTicket('userid is not followed by !')
userid = url_unquote(userid)
if '!' in data:
tokens, user_data = data.split('!', 1)
else:
# @@: Is this the right order?
tokens = ''
user_data = data
expected = calculate_digest(ip, timestamp, secret,
userid, tokens, user_data)
if expected != digest:
raise BadTicket('Digest signature is not correct',
expected=(expected, digest))
tokens = tokens.split(',')
return (timestamp, userid, tokens, user_data)
def calculate_digest(ip, timestamp, secret, userid, tokens, user_data):
secret = maybe_encode(secret)
userid = maybe_encode(userid)
tokens = maybe_encode(tokens)
user_data = maybe_encode(user_data)
digest0 = md5(
encode_ip_timestamp(ip, timestamp) + secret + userid + '\0'
+ tokens + '\0' + user_data).hexdigest()
digest = md5(digest0 + secret).hexdigest()
return digest
def encode_ip_timestamp(ip, timestamp):
ip_chars = ''.join(map(chr, map(int, ip.split('.'))))
t = int(timestamp)
ts = ((t & 0xff000000) >> 24,
(t & 0xff0000) >> 16,
(t & 0xff00) >> 8,
t & 0xff)
ts_chars = ''.join(map(chr, ts))
return ip_chars + ts_chars
def maybe_encode(s, encoding='utf8'):
if isinstance(s, unicode):
s = s.encode(encoding)
return s
class AuthTKTMiddleware(object):
"""
Middleware that checks for signed cookies that match what
`mod_auth_tkt <http://www.openfusion.com.au/labs/mod_auth_tkt/>`_
looks for (if you have mod_auth_tkt installed, you don't need this
middleware, since Apache will set the environmental variables for
you).
Arguments:
``secret``:
A secret that should be shared by any instances of this application.
If this app is served from more than one machine, they should all
have the same secret.
``cookie_name``:
The name of the cookie to read and write from. Default ``auth_tkt``.
``secure``:
If the cookie should be set as 'secure' (only sent over SSL) and if
the login must be over SSL. (Defaults to False)
``httponly``:
If the cookie should be marked as HttpOnly, which means that it's
not accessible to JavaScript. (Defaults to False)
``include_ip``:
If the cookie should include the user's IP address. If so, then
if they change IPs their cookie will be invalid.
``logout_path``:
The path under this middleware that should signify a logout. The
page will be shown as usual, but the user will also be logged out
when they visit this page.
If used with mod_auth_tkt, then these settings (except logout_path) should
match the analogous Apache configuration settings.
This also adds two functions to the request:
``environ['paste.auth_tkt.set_user'](userid, tokens='', user_data='')``
This sets a cookie that logs the user in. ``tokens`` is a
string (comma-separated groups) or a list of strings.
``user_data`` is a string for your own use.
``environ['paste.auth_tkt.logout_user']()``
Logs out the user.
"""
def __init__(self, app, secret, cookie_name='auth_tkt', secure=False,
include_ip=True, logout_path=None, httponly=False,
no_domain_cookie=True, current_domain_cookie=True,
wildcard_cookie=True):
self.app = app
self.secret = secret
self.cookie_name = cookie_name
self.secure = secure
self.httponly = httponly
self.include_ip = include_ip
self.logout_path = logout_path
self.no_domain_cookie = no_domain_cookie
self.current_domain_cookie = current_domain_cookie
self.wildcard_cookie = wildcard_cookie
def __call__(self, environ, start_response):
cookies = request.get_cookies(environ)
if self.cookie_name in cookies:
cookie_value = cookies[self.cookie_name].value
else:
cookie_value = ''
if cookie_value:
if self.include_ip:
remote_addr = environ['REMOTE_ADDR']
else:
# mod_auth_tkt uses this dummy value when IP is not
# checked:
remote_addr = '0.0.0.0'
# @@: This should handle bad signatures better:
# Also, timeouts should cause cookie refresh
try:
timestamp, userid, tokens, user_data = parse_ticket(
self.secret, cookie_value, remote_addr)
tokens = ','.join(tokens)
environ['REMOTE_USER'] = userid
if environ.get('REMOTE_USER_TOKENS'):
# We want to add tokens/roles to what's there:
tokens = environ['REMOTE_USER_TOKENS'] + ',' + tokens
environ['REMOTE_USER_TOKENS'] = tokens
environ['REMOTE_USER_DATA'] = user_data
environ['AUTH_TYPE'] = 'cookie'
except BadTicket:
# bad credentials, just ignore without logging the user
# in or anything
pass
set_cookies = []
def set_user(userid, tokens='', user_data=''):
set_cookies.extend(self.set_user_cookie(
environ, userid, tokens, user_data))
def logout_user():
set_cookies.extend(self.logout_user_cookie(environ))
environ['paste.auth_tkt.set_user'] = set_user
environ['paste.auth_tkt.logout_user'] = logout_user
if self.logout_path and environ.get('PATH_INFO') == self.logout_path:
logout_user()
def cookie_setting_start_response(status, headers, exc_info=None):
headers.extend(set_cookies)
return start_response(status, headers, exc_info)
return self.app(environ, cookie_setting_start_response)
def set_user_cookie(self, environ, userid, tokens, user_data):
if not isinstance(tokens, basestring):
tokens = ','.join(tokens)
if self.include_ip:
remote_addr = environ['REMOTE_ADDR']
else:
remote_addr = '0.0.0.0'
ticket = AuthTicket(
self.secret,
userid,
remote_addr,
tokens=tokens,
user_data=user_data,
cookie_name=self.cookie_name,
secure=self.secure)
# @@: Should we set REMOTE_USER etc in the current
# environment right now as well?
cur_domain = environ.get('HTTP_HOST', environ.get('SERVER_NAME'))
wild_domain = '.' + cur_domain
cookie_options = ""
if self.secure:
cookie_options += "; secure"
if self.httponly:
cookie_options += "; HttpOnly"
cookies = []
if self.no_domain_cookie:
cookies.append(('Set-Cookie', '%s=%s; Path=/%s' % (
self.cookie_name, ticket.cookie_value(), cookie_options)))
if self.current_domain_cookie:
cookies.append(('Set-Cookie', '%s=%s; Path=/; Domain=%s%s' % (
self.cookie_name, ticket.cookie_value(), cur_domain,
cookie_options)))
if self.wildcard_cookie:
cookies.append(('Set-Cookie', '%s=%s; Path=/; Domain=%s%s' % (
self.cookie_name, ticket.cookie_value(), wild_domain,
cookie_options)))
return cookies
def logout_user_cookie(self, environ):
cur_domain = environ.get('HTTP_HOST', environ.get('SERVER_NAME'))
wild_domain = '.' + cur_domain
expires = 'Sat, 01-Jan-2000 12:00:00 GMT'
cookies = [
('Set-Cookie', '%s=""; Expires="%s"; Path=/' % (self.cookie_name, expires)),
('Set-Cookie', '%s=""; Expires="%s"; Path=/; Domain=%s' %
(self.cookie_name, expires, cur_domain)),
('Set-Cookie', '%s=""; Expires="%s"; Path=/; Domain=%s' %
(self.cookie_name, expires, wild_domain)),
]
return cookies
def make_auth_tkt_middleware(
app,
global_conf,
secret=None,
cookie_name='auth_tkt',
secure=False,
include_ip=True,
logout_path=None):
"""
Creates the `AuthTKTMiddleware
<class-paste.auth.auth_tkt.AuthTKTMiddleware.html>`_.
``secret`` is requird, but can be set globally or locally.
"""
from paste.deploy.converters import asbool
secure = asbool(secure)
include_ip = asbool(include_ip)
if secret is None:
secret = global_conf.get('secret')
if not secret:
raise ValueError(
"You must provide a 'secret' (in global or local configuration)")
return AuthTKTMiddleware(
app, secret, cookie_name, secure, include_ip, logout_path or None)

View File

@@ -0,0 +1,122 @@
# (c) 2005 Clark C. Evans
# This module is part of the Python Paste Project and is released under
# the MIT License: http://www.opensource.org/licenses/mit-license.php
# This code was written with funding by http://prometheusresearch.com
"""
Basic HTTP/1.0 Authentication
This module implements ``Basic`` authentication as described in
HTTP/1.0 specification [1]_ . Do not use this module unless you
are using SSL or need to work with very out-dated clients, instead
use ``digest`` authentication.
>>> from paste.wsgilib import dump_environ
>>> from paste.httpserver import serve
>>> # from paste.auth.basic import AuthBasicHandler
>>> realm = 'Test Realm'
>>> def authfunc(environ, username, password):
... return username == password
>>> serve(AuthBasicHandler(dump_environ, realm, authfunc))
serving on...
.. [1] http://www.w3.org/Protocols/HTTP/1.0/draft-ietf-http-spec.html#BasicAA
"""
from paste.httpexceptions import HTTPUnauthorized
from paste.httpheaders import *
class AuthBasicAuthenticator(object):
"""
implements ``Basic`` authentication details
"""
type = 'basic'
def __init__(self, realm, authfunc):
self.realm = realm
self.authfunc = authfunc
def build_authentication(self):
head = WWW_AUTHENTICATE.tuples('Basic realm="%s"' % self.realm)
return HTTPUnauthorized(headers=head)
def authenticate(self, environ):
authorization = AUTHORIZATION(environ)
if not authorization:
return self.build_authentication()
(authmeth, auth) = authorization.split(' ', 1)
if 'basic' != authmeth.lower():
return self.build_authentication()
auth = auth.strip().decode('base64')
username, password = auth.split(':', 1)
if self.authfunc(environ, username, password):
return username
return self.build_authentication()
__call__ = authenticate
class AuthBasicHandler(object):
"""
HTTP/1.0 ``Basic`` authentication middleware
Parameters:
``application``
The application object is called only upon successful
authentication, and can assume ``environ['REMOTE_USER']``
is set. If the ``REMOTE_USER`` is already set, this
middleware is simply pass-through.
``realm``
This is a identifier for the authority that is requesting
authorization. It is shown to the user and should be unique
within the domain it is being used.
``authfunc``
This is a mandatory user-defined function which takes a
``environ``, ``username`` and ``password`` for its first
three arguments. It should return ``True`` if the user is
authenticated.
"""
def __init__(self, application, realm, authfunc):
self.application = application
self.authenticate = AuthBasicAuthenticator(realm, authfunc)
def __call__(self, environ, start_response):
username = REMOTE_USER(environ)
if not username:
result = self.authenticate(environ)
if isinstance(result, str):
AUTH_TYPE.update(environ, 'basic')
REMOTE_USER.update(environ, result)
else:
return result.wsgi_application(environ, start_response)
return self.application(environ, start_response)
middleware = AuthBasicHandler
__all__ = ['AuthBasicHandler']
def make_basic(app, global_conf, realm, authfunc, **kw):
"""
Grant access via basic authentication
Config looks like this::
[filter:grant]
use = egg:Paste#auth_basic
realm=myrealm
authfunc=somepackage.somemodule:somefunction
"""
from paste.util.import_string import eval_import
import types
authfunc = eval_import(authfunc)
assert isinstance(authfunc, types.FunctionType), "authfunc must resolve to a function"
return AuthBasicHandler(app, realm, authfunc)
if "__main__" == __name__:
import doctest
doctest.testmod(optionflags=doctest.ELLIPSIS)

View File

@@ -0,0 +1,99 @@
# (c) 2005 Clark C. Evans
# This module is part of the Python Paste Project and is released under
# the MIT License: http://www.opensource.org/licenses/mit-license.php
# This code was written with funding by http://prometheusresearch.com
"""
CAS 1.0 Authentication
The Central Authentication System is a straight-forward single sign-on
mechanism developed by Yale University's ITS department. It has since
enjoyed widespread success and is deployed at many major universities
and some corporations.
https://clearinghouse.ja-sig.org/wiki/display/CAS/Home
http://www.yale.edu/tp/auth/usingcasatyale.html
This implementation has the goal of maintaining current path arguments
passed to the system so that it can be used as middleware at any stage
of processing. It has the secondary goal of allowing for other
authentication methods to be used concurrently.
"""
import urllib
from paste.request import construct_url
from paste.httpexceptions import HTTPSeeOther, HTTPForbidden
class CASLoginFailure(HTTPForbidden):
""" The exception raised if the authority returns 'no' """
class CASAuthenticate(HTTPSeeOther):
""" The exception raised to authenticate the user """
def AuthCASHandler(application, authority):
"""
middleware to implement CAS 1.0 authentication
There are several possible outcomes:
0. If the REMOTE_USER environment variable is already populated;
then this middleware is a no-op, and the request is passed along
to the application.
1. If a query argument 'ticket' is found, then an attempt to
validate said ticket /w the authentication service done. If the
ticket is not validated; an 403 'Forbidden' exception is raised.
Otherwise, the REMOTE_USER variable is set with the NetID that
was validated and AUTH_TYPE is set to "cas".
2. Otherwise, a 303 'See Other' is returned to the client directing
them to login using the CAS service. After logon, the service
will send them back to this same URL, only with a 'ticket' query
argument.
Parameters:
``authority``
This is a fully-qualified URL to a CAS 1.0 service. The URL
should end with a '/' and have the 'login' and 'validate'
sub-paths as described in the CAS 1.0 documentation.
"""
assert authority.endswith("/") and authority.startswith("http")
def cas_application(environ, start_response):
username = environ.get('REMOTE_USER','')
if username:
return application(environ, start_response)
qs = environ.get('QUERY_STRING','').split("&")
if qs and qs[-1].startswith("ticket="):
# assume a response from the authority
ticket = qs.pop().split("=", 1)[1]
environ['QUERY_STRING'] = "&".join(qs)
service = construct_url(environ)
args = urllib.urlencode(
{'service': service,'ticket': ticket})
requrl = authority + "validate?" + args
result = urllib.urlopen(requrl).read().split("\n")
if 'yes' == result[0]:
environ['REMOTE_USER'] = result[1]
environ['AUTH_TYPE'] = 'cas'
return application(environ, start_response)
exce = CASLoginFailure()
else:
service = construct_url(environ)
args = urllib.urlencode({'service': service})
location = authority + "login?" + args
exce = CASAuthenticate(location)
return exce.wsgi_application(environ, start_response)
return cas_application
middleware = AuthCASHandler
__all__ = ['CASLoginFailure', 'CASAuthenticate', 'AuthCASHandler' ]
if '__main__' == __name__:
authority = "https://secure.its.yale.edu/cas/servlet/"
from paste.wsgilib import dump_environ
from paste.httpserver import serve
from paste.httpexceptions import *
serve(HTTPExceptionHandler(
AuthCASHandler(dump_environ, authority)))

View File

@@ -0,0 +1,396 @@
# (c) 2005 Clark C. Evans
# This module is part of the Python Paste Project and is released under
# the MIT License: http://www.opensource.org/licenses/mit-license.php
# This code was written with funding by http://prometheusresearch.com
"""
Cookie "Saved" Authentication
This authentication middleware saves the current REMOTE_USER,
REMOTE_SESSION, and any other environment variables specified in a
cookie so that it can be retrieved during the next request without
requiring re-authentication. This uses a session cookie on the client
side (so it goes away when the user closes their window) and does
server-side expiration.
Following is a very simple example where a form is presented asking for
a user name (no actual checking), and dummy session identifier (perhaps
corresponding to a database session id) is stored in the cookie.
::
>>> from paste.httpserver import serve
>>> from paste.fileapp import DataApp
>>> from paste.httpexceptions import *
>>> from paste.auth.cookie import AuthCookieHandler
>>> from paste.wsgilib import parse_querystring
>>> def testapp(environ, start_response):
... user = dict(parse_querystring(environ)).get('user','')
... if user:
... environ['REMOTE_USER'] = user
... environ['REMOTE_SESSION'] = 'a-session-id'
... if environ.get('REMOTE_USER'):
... page = '<html><body>Welcome %s (%s)</body></html>'
... page %= (environ['REMOTE_USER'], environ['REMOTE_SESSION'])
... else:
... page = ('<html><body><form><input name="user" />'
... '<input type="submit" /></form></body></html>')
... return DataApp(page, content_type="text/html")(
... environ, start_response)
>>> serve(AuthCookieHandler(testapp))
serving on...
"""
import hmac, base64, random, time, warnings
try:
from hashlib import sha1
except ImportError:
# NOTE: We have to use the callable with hashlib (hashlib.sha1),
# otherwise hmac only accepts the sha module object itself
import sha as sha1
from paste.request import get_cookies
def make_time(value):
return time.strftime("%Y%m%d%H%M", time.gmtime(value))
_signature_size = len(hmac.new('x', 'x', sha1).digest())
_header_size = _signature_size + len(make_time(time.time()))
# @@: Should this be using urllib.quote?
# build encode/decode functions to safely pack away values
_encode = [('\\', '\\x5c'), ('"', '\\x22'),
('=', '\\x3d'), (';', '\\x3b')]
_decode = [(v, k) for (k, v) in _encode]
_decode.reverse()
def encode(s, sublist = _encode):
return reduce((lambda a, (b, c): a.replace(b, c)), sublist, str(s))
decode = lambda s: encode(s, _decode)
class CookieTooLarge(RuntimeError):
def __init__(self, content, cookie):
RuntimeError.__init__("Signed cookie exceeds maximum size of 4096")
self.content = content
self.cookie = cookie
_all_chars = ''.join([chr(x) for x in range(0, 255)])
def new_secret():
""" returns a 64 byte secret """
return ''.join(random.sample(_all_chars, 64))
class AuthCookieSigner(object):
"""
save/restore ``environ`` entries via digially signed cookie
This class converts content into a timed and digitally signed
cookie, as well as having the facility to reverse this procedure.
If the cookie, after the content is encoded and signed exceeds the
maximum length (4096), then CookieTooLarge exception is raised.
The timeout of the cookie is handled on the server side for a few
reasons. First, if a 'Expires' directive is added to a cookie, then
the cookie becomes persistent (lasting even after the browser window
has closed). Second, the user's clock may be wrong (perhaps
intentionally). The timeout is specified in minutes; and expiration
date returned is rounded to one second.
Constructor Arguments:
``secret``
This is a secret key if you want to syncronize your keys so
that the cookie will be good across a cluster of computers.
It is recommended via the HMAC specification (RFC 2104) that
the secret key be 64 bytes since this is the block size of
the hashing. If you do not provide a secret key, a random
one is generated each time you create the handler; this
should be sufficient for most cases.
``timeout``
This is the time (in minutes) from which the cookie is set
to expire. Note that on each request a new (replacement)
cookie is sent, hence this is effectively a session timeout
parameter for your entire cluster. If you do not provide a
timeout, it is set at 30 minutes.
``maxlen``
This is the maximum size of the *signed* cookie; hence the
actual content signed will be somewhat less. If the cookie
goes over this size, a ``CookieTooLarge`` exception is
raised so that unexpected handling of cookies on the client
side are avoided. By default this is set at 4k (4096 bytes),
which is the standard cookie size limit.
"""
def __init__(self, secret = None, timeout = None, maxlen = None):
self.timeout = timeout or 30
if isinstance(timeout, basestring):
raise ValueError(
"Timeout must be a number (minutes), not a string (%r)"
% timeout)
self.maxlen = maxlen or 4096
self.secret = secret or new_secret()
def sign(self, content):
"""
Sign the content returning a valid cookie (that does not
need to be escaped and quoted). The expiration of this
cookie is handled server-side in the auth() function.
"""
cookie = base64.encodestring(
hmac.new(self.secret, content, sha1).digest() +
make_time(time.time() + 60*self.timeout) +
content)
cookie = cookie.replace("/", "_").replace("=", "~")
cookie = cookie.replace('\n', '').replace('\r', '')
if len(cookie) > self.maxlen:
raise CookieTooLarge(content, cookie)
return cookie
def auth(self, cookie):
"""
Authenticate the cooke using the signature, verify that it
has not expired; and return the cookie's content
"""
decode = base64.decodestring(
cookie.replace("_", "/").replace("~", "="))
signature = decode[:_signature_size]
expires = decode[_signature_size:_header_size]
content = decode[_header_size:]
if signature == hmac.new(self.secret, content, sha1).digest():
if int(expires) > int(make_time(time.time())):
return content
else:
# This is the normal case of an expired cookie; just
# don't bother doing anything here.
pass
else:
# This case can happen if the server is restarted with a
# different secret; or if the user's IP address changed
# due to a proxy. However, it could also be a break-in
# attempt -- so should it be reported?
pass
class AuthCookieEnviron(list):
"""
a list of environment keys to be saved via cookie
An instance of this object, found at ``environ['paste.auth.cookie']``
lists the `environ` keys that were restored from or will be added
to the digially signed cookie. This object can be accessed from an
`environ` variable by using this module's name.
"""
def __init__(self, handler, scanlist):
list.__init__(self, scanlist)
self.handler = handler
def append(self, value):
if value in self:
return
list.append(self, str(value))
class AuthCookieHandler(object):
"""
the actual handler that should be put in your middleware stack
This middleware uses cookies to stash-away a previously authenticated
user (and perhaps other variables) so that re-authentication is not
needed. This does not implement sessions; and therefore N servers
can be syncronized to accept the same saved authentication if they
all use the same cookie_name and secret.
By default, this handler scans the `environ` for the REMOTE_USER
and REMOTE_SESSION key; if found, it is stored. It can be
configured to scan other `environ` keys as well -- but be careful
not to exceed 2-3k (so that the encoded and signed cookie does not
exceed 4k). You can ask it to handle other environment variables
by doing:
``environ['paste.auth.cookie'].append('your.environ.variable')``
Constructor Arguments:
``application``
This is the wrapped application which will have access to
the ``environ['REMOTE_USER']`` restored by this middleware.
``cookie_name``
The name of the cookie used to store this content, by default
it is ``PASTE_AUTH_COOKIE``.
``scanlist``
This is the initial set of ``environ`` keys to
save/restore to the signed cookie. By default is consists
only of ``REMOTE_USER`` and ``REMOTE_SESSION``; any tuple
or list of environment keys will work. However, be
careful, as the total saved size is limited to around 3k.
``signer``
This is the signer object used to create the actual cookie
values, by default, it is ``AuthCookieSigner`` and is passed
the remaining arguments to this function: ``secret``,
``timeout``, and ``maxlen``.
At this time, each cookie is individually signed. To store more
than the 4k of data; it is possible to sub-class this object to
provide different ``environ_name`` and ``cookie_name``
"""
environ_name = 'paste.auth.cookie'
cookie_name = 'PASTE_AUTH_COOKIE'
signer_class = AuthCookieSigner
environ_class = AuthCookieEnviron
def __init__(self, application, cookie_name=None, scanlist=None,
signer=None, secret=None, timeout=None, maxlen=None):
if not signer:
signer = self.signer_class(secret, timeout, maxlen)
self.signer = signer
self.scanlist = scanlist or ('REMOTE_USER','REMOTE_SESSION')
self.application = application
self.cookie_name = cookie_name or self.cookie_name
def __call__(self, environ, start_response):
if self.environ_name in environ:
raise AssertionError("AuthCookie already installed!")
scanlist = self.environ_class(self, self.scanlist)
jar = get_cookies(environ)
if jar.has_key(self.cookie_name):
content = self.signer.auth(jar[self.cookie_name].value)
if content:
for pair in content.split(";"):
(k, v) = pair.split("=")
k = decode(k)
if k not in scanlist:
scanlist.append(k)
if k in environ:
continue
environ[k] = decode(v)
if 'REMOTE_USER' == k:
environ['AUTH_TYPE'] = 'cookie'
environ[self.environ_name] = scanlist
if "paste.httpexceptions" in environ:
warnings.warn("Since paste.httpexceptions is hooked in your "
"processing chain before paste.auth.cookie, if an "
"HTTPRedirection is raised, the cookies this module sets "
"will not be included in your response.\n")
def response_hook(status, response_headers, exc_info=None):
"""
Scan the environment for keys specified in the scanlist,
pack up their values, signs the content and issues a cookie.
"""
scanlist = environ.get(self.environ_name)
assert scanlist and isinstance(scanlist, self.environ_class)
content = []
for k in scanlist:
v = environ.get(k)
if v is not None:
if type(v) is not str:
raise ValueError(
"The value of the environmental variable %r "
"is not a str (only str is allowed; got %r)"
% (k, v))
content.append("%s=%s" % (encode(k), encode(v)))
if content:
content = ";".join(content)
content = self.signer.sign(content)
cookie = '%s=%s; Path=/;' % (self.cookie_name, content)
if 'https' == environ['wsgi.url_scheme']:
cookie += ' secure;'
response_headers.append(('Set-Cookie', cookie))
return start_response(status, response_headers, exc_info)
return self.application(environ, response_hook)
middleware = AuthCookieHandler
# Paste Deploy entry point:
def make_auth_cookie(
app, global_conf,
# Should this get picked up from global_conf somehow?:
cookie_name='PASTE_AUTH_COOKIE',
scanlist=('REMOTE_USER', 'REMOTE_SESSION'),
# signer cannot be set
secret=None,
timeout=30,
maxlen=4096):
"""
This middleware uses cookies to stash-away a previously
authenticated user (and perhaps other variables) so that
re-authentication is not needed. This does not implement
sessions; and therefore N servers can be syncronized to accept the
same saved authentication if they all use the same cookie_name and
secret.
By default, this handler scans the `environ` for the REMOTE_USER
and REMOTE_SESSION key; if found, it is stored. It can be
configured to scan other `environ` keys as well -- but be careful
not to exceed 2-3k (so that the encoded and signed cookie does not
exceed 4k). You can ask it to handle other environment variables
by doing:
``environ['paste.auth.cookie'].append('your.environ.variable')``
Configuration:
``cookie_name``
The name of the cookie used to store this content, by
default it is ``PASTE_AUTH_COOKIE``.
``scanlist``
This is the initial set of ``environ`` keys to
save/restore to the signed cookie. By default is consists
only of ``REMOTE_USER`` and ``REMOTE_SESSION``; any
space-separated list of environment keys will work.
However, be careful, as the total saved size is limited to
around 3k.
``secret``
The secret that will be used to sign the cookies. If you
don't provide one (and none is set globally) then a random
secret will be created. Each time the server is restarted
a new secret will then be created and all cookies will
become invalid! This can be any string value.
``timeout``
The time to keep the cookie, expressed in minutes. This
is handled server-side, so a new cookie with a new timeout
is added to every response.
``maxlen``
The maximum length of the cookie that is sent (default 4k,
which is a typical browser maximum)
"""
if isinstance(scanlist, basestring):
scanlist = scanlist.split()
if secret is None and global_conf.get('secret'):
secret = global_conf['secret']
try:
timeout = int(timeout)
except ValueError:
raise ValueError('Bad value for timeout (must be int): %r'
% timeout)
try:
maxlen = int(maxlen)
except ValueError:
raise ValueError('Bad value for maxlen (must be int): %r'
% maxlen)
return AuthCookieHandler(
app, cookie_name=cookie_name, scanlist=scanlist,
secret=secret, timeout=timeout, maxlen=maxlen)
__all__ = ['AuthCookieHandler', 'AuthCookieSigner', 'AuthCookieEnviron']
if "__main__" == __name__:
import doctest
doctest.testmod(optionflags=doctest.ELLIPSIS)

View File

@@ -0,0 +1,214 @@
# (c) 2005 Clark C. Evans
# This module is part of the Python Paste Project and is released under
# the MIT License: http://www.opensource.org/licenses/mit-license.php
# This code was written with funding by http://prometheusresearch.com
"""
Digest HTTP/1.1 Authentication
This module implements ``Digest`` authentication as described by
RFC 2617 [1]_ .
Basically, you just put this module before your application, and it
takes care of requesting and handling authentication requests. This
module has been tested with several common browsers "out-in-the-wild".
>>> from paste.wsgilib import dump_environ
>>> from paste.httpserver import serve
>>> # from paste.auth.digest import digest_password, AuthDigestHandler
>>> realm = 'Test Realm'
>>> def authfunc(environ, realm, username):
... return digest_password(realm, username, username)
>>> serve(AuthDigestHandler(dump_environ, realm, authfunc))
serving on...
This code has not been audited by a security expert, please use with
caution (or better yet, report security holes). At this time, this
implementation does not provide for further challenges, nor does it
support Authentication-Info header. It also uses md5, and an option
to use sha would be a good thing.
.. [1] http://www.faqs.org/rfcs/rfc2617.html
"""
from paste.httpexceptions import HTTPUnauthorized
from paste.httpheaders import *
try:
from hashlib import md5
except ImportError:
from md5 import md5
import time, random
from urllib import quote as url_quote
def digest_password(realm, username, password):
""" construct the appropriate hashcode needed for HTTP digest """
return md5("%s:%s:%s" % (username, realm, password)).hexdigest()
class AuthDigestAuthenticator(object):
""" implementation of RFC 2617 - HTTP Digest Authentication """
def __init__(self, realm, authfunc):
self.nonce = {} # list to prevent replay attacks
self.authfunc = authfunc
self.realm = realm
def build_authentication(self, stale = ''):
""" builds the authentication error """
nonce = md5(
"%s:%s" % (time.time(), random.random())).hexdigest()
opaque = md5(
"%s:%s" % (time.time(), random.random())).hexdigest()
self.nonce[nonce] = None
parts = {'realm': self.realm, 'qop': 'auth',
'nonce': nonce, 'opaque': opaque }
if stale:
parts['stale'] = 'true'
head = ", ".join(['%s="%s"' % (k, v) for (k, v) in parts.items()])
head = [("WWW-Authenticate", 'Digest %s' % head)]
return HTTPUnauthorized(headers=head)
def compute(self, ha1, username, response, method,
path, nonce, nc, cnonce, qop):
""" computes the authentication, raises error if unsuccessful """
if not ha1:
return self.build_authentication()
ha2 = md5('%s:%s' % (method, path)).hexdigest()
if qop:
chk = "%s:%s:%s:%s:%s:%s" % (ha1, nonce, nc, cnonce, qop, ha2)
else:
chk = "%s:%s:%s" % (ha1, nonce, ha2)
if response != md5(chk).hexdigest():
if nonce in self.nonce:
del self.nonce[nonce]
return self.build_authentication()
pnc = self.nonce.get(nonce,'00000000')
if nc <= pnc:
if nonce in self.nonce:
del self.nonce[nonce]
return self.build_authentication(stale = True)
self.nonce[nonce] = nc
return username
def authenticate(self, environ):
""" This function takes a WSGI environment and authenticates
the request returning authenticated user or error.
"""
method = REQUEST_METHOD(environ)
fullpath = url_quote(SCRIPT_NAME(environ)) + url_quote(PATH_INFO(environ))
authorization = AUTHORIZATION(environ)
if not authorization:
return self.build_authentication()
(authmeth, auth) = authorization.split(" ", 1)
if 'digest' != authmeth.lower():
return self.build_authentication()
amap = {}
for itm in auth.split(", "):
(k,v) = [s.strip() for s in itm.split("=", 1)]
amap[k] = v.replace('"', '')
try:
username = amap['username']
authpath = amap['uri']
nonce = amap['nonce']
realm = amap['realm']
response = amap['response']
assert authpath.split("?", 1)[0] in fullpath
assert realm == self.realm
qop = amap.get('qop', '')
cnonce = amap.get('cnonce', '')
nc = amap.get('nc', '00000000')
if qop:
assert 'auth' == qop
assert nonce and nc
except:
return self.build_authentication()
ha1 = self.authfunc(environ, realm, username)
return self.compute(ha1, username, response, method, authpath,
nonce, nc, cnonce, qop)
__call__ = authenticate
class AuthDigestHandler(object):
"""
middleware for HTTP Digest authentication (RFC 2617)
This component follows the procedure below:
0. If the REMOTE_USER environment variable is already populated;
then this middleware is a no-op, and the request is passed
along to the application.
1. If the HTTP_AUTHORIZATION header was not provided or specifies
an algorithem other than ``digest``, then a HTTPUnauthorized
response is generated with the challenge.
2. If the response is malformed or or if the user's credientials
do not pass muster, another HTTPUnauthorized is raised.
3. If all goes well, and the user's credintials pass; then
REMOTE_USER environment variable is filled in and the
AUTH_TYPE is listed as 'digest'.
Parameters:
``application``
The application object is called only upon successful
authentication, and can assume ``environ['REMOTE_USER']``
is set. If the ``REMOTE_USER`` is already set, this
middleware is simply pass-through.
``realm``
This is a identifier for the authority that is requesting
authorization. It is shown to the user and should be unique
within the domain it is being used.
``authfunc``
This is a callback function which performs the actual
authentication; the signature of this callback is:
authfunc(environ, realm, username) -> hashcode
This module provides a 'digest_password' helper function
which can help construct the hashcode; it is recommended
that the hashcode is stored in a database, not the user's
actual password (since you only need the hashcode).
"""
def __init__(self, application, realm, authfunc):
self.authenticate = AuthDigestAuthenticator(realm, authfunc)
self.application = application
def __call__(self, environ, start_response):
username = REMOTE_USER(environ)
if not username:
result = self.authenticate(environ)
if isinstance(result, str):
AUTH_TYPE.update(environ,'digest')
REMOTE_USER.update(environ, result)
else:
return result.wsgi_application(environ, start_response)
return self.application(environ, start_response)
middleware = AuthDigestHandler
__all__ = ['digest_password', 'AuthDigestHandler' ]
def make_digest(app, global_conf, realm, authfunc, **kw):
"""
Grant access via digest authentication
Config looks like this::
[filter:grant]
use = egg:Paste#auth_digest
realm=myrealm
authfunc=somepackage.somemodule:somefunction
"""
from paste.util.import_string import eval_import
import types
authfunc = eval_import(authfunc)
assert isinstance(authfunc, types.FunctionType), "authfunc must resolve to a function"
return AuthDigestHandler(app, realm, authfunc)
if "__main__" == __name__:
import doctest
doctest.testmod(optionflags=doctest.ELLIPSIS)

View File

@@ -0,0 +1,149 @@
# (c) 2005 Clark C. Evans
# This module is part of the Python Paste Project and is released under
# the MIT License: http://www.opensource.org/licenses/mit-license.php
# This code was written with funding by http://prometheusresearch.com
"""
Authentication via HTML Form
This is a very simple HTML form login screen that asks for the username
and password. This middleware component requires that an authorization
function taking the name and passsword and that it be placed in your
application stack. This class does not include any session management
code or way to save the user's authorization; however, it is easy enough
to put ``paste.auth.cookie`` in your application stack.
>>> from paste.wsgilib import dump_environ
>>> from paste.httpserver import serve
>>> from paste.auth.cookie import AuthCookieHandler
>>> from paste.auth.form import AuthFormHandler
>>> def authfunc(environ, username, password):
... return username == password
>>> serve(AuthCookieHandler(
... AuthFormHandler(dump_environ, authfunc)))
serving on...
"""
from paste.request import construct_url, parse_formvars
TEMPLATE = """\
<html>
<head><title>Please Login!</title></head>
<body>
<h1>Please Login</h1>
<form action="%s" method="post">
<dl>
<dt>Username:</dt>
<dd><input type="text" name="username"></dd>
<dt>Password:</dt>
<dd><input type="password" name="password"></dd>
</dl>
<input type="submit" name="authform" />
<hr />
</form>
</body>
</html>
"""
class AuthFormHandler(object):
"""
HTML-based login middleware
This causes a HTML form to be returned if ``REMOTE_USER`` is
not found in the ``environ``. If the form is returned, the
``username`` and ``password`` combination are given to a
user-supplied authentication function, ``authfunc``. If this
is successful, then application processing continues.
Parameters:
``application``
The application object is called only upon successful
authentication, and can assume ``environ['REMOTE_USER']``
is set. If the ``REMOTE_USER`` is already set, this
middleware is simply pass-through.
``authfunc``
This is a mandatory user-defined function which takes a
``environ``, ``username`` and ``password`` for its first
three arguments. It should return ``True`` if the user is
authenticated.
``template``
This is an optional (a default is provided) HTML
fragment that takes exactly one ``%s`` substution
argument; which *must* be used for the form's ``action``
to ensure that this middleware component does not alter
the current path. The HTML form must use ``POST`` and
have two input names: ``username`` and ``password``.
Since the authentication form is submitted (via ``POST``)
neither the ``PATH_INFO`` nor the ``QUERY_STRING`` are accessed,
and hence the current path remains _unaltered_ through the
entire authentication process. If authentication succeeds, the
``REQUEST_METHOD`` is converted from a ``POST`` to a ``GET``,
so that a redirect is unnecessary (unlike most form auth
implementations)
"""
def __init__(self, application, authfunc, template=None):
self.application = application
self.authfunc = authfunc
self.template = template or TEMPLATE
def __call__(self, environ, start_response):
username = environ.get('REMOTE_USER','')
if username:
return self.application(environ, start_response)
if 'POST' == environ['REQUEST_METHOD']:
formvars = parse_formvars(environ, include_get_vars=False)
username = formvars.get('username')
password = formvars.get('password')
if username and password:
if self.authfunc(environ, username, password):
environ['AUTH_TYPE'] = 'form'
environ['REMOTE_USER'] = username
environ['REQUEST_METHOD'] = 'GET'
environ['CONTENT_LENGTH'] = ''
environ['CONTENT_TYPE'] = ''
del environ['paste.parsed_formvars']
return self.application(environ, start_response)
content = self.template % construct_url(environ)
start_response("200 OK", [('Content-Type', 'text/html'),
('Content-Length', str(len(content)))])
return [content]
middleware = AuthFormHandler
__all__ = ['AuthFormHandler']
def make_form(app, global_conf, realm, authfunc, **kw):
"""
Grant access via form authentication
Config looks like this::
[filter:grant]
use = egg:Paste#auth_form
realm=myrealm
authfunc=somepackage.somemodule:somefunction
"""
from paste.util.import_string import eval_import
import types
authfunc = eval_import(authfunc)
assert isinstance(authfunc, types.FunctionType), "authfunc must resolve to a function"
template = kw.get('template')
if template is not None:
template = eval_import(template)
assert isinstance(template, str), "template must resolve to a string"
return AuthFormHandler(app, authfunc, template)
if "__main__" == __name__:
import doctest
doctest.testmod(optionflags=doctest.ELLIPSIS)

View File

@@ -0,0 +1,113 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
"""
Grant roles and logins based on IP address.
"""
from paste.util import ip4
class GrantIPMiddleware(object):
"""
On each request, ``ip_map`` is checked against ``REMOTE_ADDR``
and logins and roles are assigned based on that.
``ip_map`` is a map of {ip_mask: (username, roles)}. Either
``username`` or ``roles`` may be None. Roles may also be prefixed
with ``-``, like ``'-system'`` meaning that role should be
revoked. ``'__remove__'`` for a username will remove the username.
If ``clobber_username`` is true (default) then any user
specification will override the current value of ``REMOTE_USER``.
``'__remove__'`` will always clobber the username.
``ip_mask`` is something that `paste.util.ip4:IP4Range
<class-paste.util.ip4.IP4Range.html>`_ can parse. Simple IP
addresses, IP/mask, ip<->ip ranges, and hostnames are allowed.
"""
def __init__(self, app, ip_map, clobber_username=True):
self.app = app
self.ip_map = []
for key, value in ip_map.items():
self.ip_map.append((ip4.IP4Range(key),
self._convert_user_role(value[0], value[1])))
self.clobber_username = clobber_username
def _convert_user_role(self, username, roles):
if roles and isinstance(roles, basestring):
roles = roles.split(',')
return (username, roles)
def __call__(self, environ, start_response):
addr = ip4.ip2int(environ['REMOTE_ADDR'], False)
remove_user = False
add_roles = []
for range, (username, roles) in self.ip_map:
if addr in range:
if roles:
add_roles.extend(roles)
if username == '__remove__':
remove_user = True
elif username:
if (not environ.get('REMOTE_USER')
or self.clobber_username):
environ['REMOTE_USER'] = username
if (remove_user and 'REMOTE_USER' in environ):
del environ['REMOTE_USER']
if roles:
self._set_roles(environ, add_roles)
return self.app(environ, start_response)
def _set_roles(self, environ, roles):
cur_roles = environ.get('REMOTE_USER_TOKENS', '').split(',')
# Get rid of empty roles:
cur_roles = filter(None, cur_roles)
remove_roles = []
for role in roles:
if role.startswith('-'):
remove_roles.append(role[1:])
else:
if role not in cur_roles:
cur_roles.append(role)
for role in remove_roles:
if role in cur_roles:
cur_roles.remove(role)
environ['REMOTE_USER_TOKENS'] = ','.join(cur_roles)
def make_grantip(app, global_conf, clobber_username=False, **kw):
"""
Grant roles or usernames based on IP addresses.
Config looks like this::
[filter:grant]
use = egg:Paste#grantip
clobber_username = true
# Give localhost system role (no username):
127.0.0.1 = -:system
# Give everyone in 192.168.0.* editor role:
192.168.0.0/24 = -:editor
# Give one IP the username joe:
192.168.0.7 = joe
# And one IP is should not be logged in:
192.168.0.10 = __remove__:-editor
"""
from paste.deploy.converters import asbool
clobber_username = asbool(clobber_username)
ip_map = {}
for key, value in kw.items():
if ':' in value:
username, role = value.split(':', 1)
else:
username = value
role = ''
if username == '-':
username = ''
if role == '-':
role = ''
ip_map[key] = value
return GrantIPMiddleware(app, ip_map, clobber_username)

View File

@@ -0,0 +1,79 @@
# (c) 2005 Clark C. Evans
# This module is part of the Python Paste Project and is released under
# the MIT License: http://www.opensource.org/licenses/mit-license.php
# This code was written with funding by http://prometheusresearch.com
"""
Authentication via Multiple Methods
In some environments, the choice of authentication method to be used
depends upon the environment and is not "fixed". This middleware allows
N authentication methods to be registered along with a goodness function
which determines which method should be used. The following example
demonstrates how to use both form and digest authentication in a server
stack; by default it uses form-based authentication unless
``*authmeth=digest`` is specified as a query argument.
>>> from paste.auth import form, cookie, digest, multi
>>> from paste.wsgilib import dump_environ
>>> from paste.httpserver import serve
>>>
>>> multi = multi.MultiHandler(dump_environ)
>>> def authfunc(environ, realm, user):
... return digest.digest_password(realm, user, user)
>>> multi.add_method('digest', digest.middleware, "Test Realm", authfunc)
>>> multi.set_query_argument('digest')
>>>
>>> def authfunc(environ, username, password):
... return username == password
>>> multi.add_method('form', form.middleware, authfunc)
>>> multi.set_default('form')
>>> serve(cookie.middleware(multi))
serving on...
"""
class MultiHandler(object):
"""
Multiple Authentication Handler
This middleware provides two othogonal facilities:
- a manner to register any number of authentication middlewares
- a mechanism to register predicates which cause one of the
registered middlewares to be used depending upon the request
If none of the predicates returns True, then the application is
invoked directly without middleware
"""
def __init__(self, application):
self.application = application
self.default = application
self.binding = {}
self.predicate = []
def add_method(self, name, factory, *args, **kwargs):
self.binding[name] = factory(self.application, *args, **kwargs)
def add_predicate(self, name, checker):
self.predicate.append((checker, self.binding[name]))
def set_default(self, name):
""" set default authentication method """
self.default = self.binding[name]
def set_query_argument(self, name, key = '*authmeth', value = None):
""" choose authentication method based on a query argument """
lookfor = "%s=%s" % (key, value or name)
self.add_predicate(name,
lambda environ: lookfor in environ.get('QUERY_STRING',''))
def __call__(self, environ, start_response):
for (checker, binding) in self.predicate:
if checker(environ):
return binding(environ, start_response)
return self.default(environ, start_response)
middleware = MultiHandler
__all__ = ['MultiHandler']
if "__main__" == __name__:
import doctest
doctest.testmod(optionflags=doctest.ELLIPSIS)

View File

@@ -0,0 +1,412 @@
# (c) 2005 Ben Bangert
# This module is part of the Python Paste Project and is released under
# the MIT License: http://www.opensource.org/licenses/mit-license.php
"""
OpenID Authentication (Consumer)
OpenID is a distributed authentication system for single sign-on originally
developed at/for LiveJournal.com.
http://openid.net/
URL. You can have multiple identities in the same way you can have multiple
URLs. All OpenID does is provide a way to prove that you own a URL (identity).
And it does this without passing around your password, your email address, or
anything you don't want it to. There's no profile exchange component at all:
your profiile is your identity URL, but recipients of your identity can then
learn more about you from any public, semantically interesting documents
linked thereunder (FOAF, RSS, Atom, vCARD, etc.).
``Note``: paste.auth.openid requires installation of the Python-OpenID
libraries::
http://www.openidenabled.com/
This module is based highly off the consumer.py that Python OpenID comes with.
Using the OpenID Middleware
===========================
Using the OpenID middleware is fairly easy, the most minimal example using the
basic login form thats included::
# Add to your wsgi app creation
from paste.auth import open_id
wsgi_app = open_id.middleware(wsgi_app, '/somewhere/to/store/openid/data')
You will now have the OpenID form available at /oid on your site. Logging in will
verify that the login worked.
A more complete login should involve having the OpenID middleware load your own
login page after verifying the OpenID URL so that you can retain the login
information in your webapp (session, cookies, etc.)::
wsgi_app = open_id.middleware(wsgi_app, '/somewhere/to/store/openid/data',
login_redirect='/your/login/code')
Your login code should then be configured to retrieve 'paste.auth.open_id' for
the users OpenID URL. If this key does not exist, the user has not logged in.
Once the login is retrieved, it should be saved in your webapp, and the user
should be redirected to wherever they would normally go after a successful
login.
"""
__all__ = ['AuthOpenIDHandler']
import cgi
import urlparse
import re
import paste.request
from paste import httpexceptions
def quoteattr(s):
qs = cgi.escape(s, 1)
return '"%s"' % (qs,)
# You may need to manually add the openid package into your
# python path if you don't have it installed with your system python.
# If so, uncomment the line below, and change the path where you have
# Python-OpenID.
# sys.path.append('/path/to/openid/')
from openid.store import filestore
from openid.consumer import consumer
from openid.oidutil import appendArgs
class AuthOpenIDHandler(object):
"""
This middleware implements OpenID Consumer behavior to authenticate a
URL against an OpenID Server.
"""
def __init__(self, app, data_store_path, auth_prefix='/oid',
login_redirect=None, catch_401=False,
url_to_username=None):
"""
Initialize the OpenID middleware
``app``
Your WSGI app to call
``data_store_path``
Directory to store crypto data in for use with OpenID servers.
``auth_prefix``
Location for authentication process/verification
``login_redirect``
Location to load after successful process of login
``catch_401``
If true, then any 401 responses will turn into open ID login
requirements.
``url_to_username``
A function called like ``url_to_username(environ, url)``, which should
return a string username. If not given, the URL will be the username.
"""
store = filestore.FileOpenIDStore(data_store_path)
self.oidconsumer = consumer.OpenIDConsumer(store)
self.app = app
self.auth_prefix = auth_prefix
self.data_store_path = data_store_path
self.login_redirect = login_redirect
self.catch_401 = catch_401
self.url_to_username = url_to_username
def __call__(self, environ, start_response):
if environ['PATH_INFO'].startswith(self.auth_prefix):
# Let's load everything into a request dict to pass around easier
request = dict(environ=environ, start=start_response, body=[])
request['base_url'] = paste.request.construct_url(environ, with_path_info=False,
with_query_string=False)
path = re.sub(self.auth_prefix, '', environ['PATH_INFO'])
request['parsed_uri'] = urlparse.urlparse(path)
request['query'] = dict(paste.request.parse_querystring(environ))
path = request['parsed_uri'][2]
if path == '/' or not path:
return self.render(request)
elif path == '/verify':
return self.do_verify(request)
elif path == '/process':
return self.do_process(request)
else:
return self.not_found(request)
else:
if self.catch_401:
return self.catch_401_app_call(environ, start_response)
return self.app(environ, start_response)
def catch_401_app_call(self, environ, start_response):
"""
Call the application, and redirect if the app returns a 401 response
"""
was_401 = []
def replacement_start_response(status, headers, exc_info=None):
if int(status.split(None, 1)) == 401:
# @@: Do I need to append something to go back to where we
# came from?
was_401.append(1)
def dummy_writer(v):
pass
return dummy_writer
else:
return start_response(status, headers, exc_info)
app_iter = self.app(environ, replacement_start_response)
if was_401:
try:
list(app_iter)
finally:
if hasattr(app_iter, 'close'):
app_iter.close()
redir_url = paste.request.construct_url(environ, with_path_info=False,
with_query_string=False)
exc = httpexceptions.HTTPTemporaryRedirect(redir_url)
return exc.wsgi_application(environ, start_response)
else:
return app_iter
def do_verify(self, request):
"""Process the form submission, initating OpenID verification.
"""
# First, make sure that the user entered something
openid_url = request['query'].get('openid_url')
if not openid_url:
return self.render(request, 'Enter an identity URL to verify.',
css_class='error', form_contents=openid_url)
oidconsumer = self.oidconsumer
# Then, ask the library to begin the authorization.
# Here we find out the identity server that will verify the
# user's identity, and get a token that allows us to
# communicate securely with the identity server.
status, info = oidconsumer.beginAuth(openid_url)
# If the URL was unusable (either because of network
# conditions, a server error, or that the response returned
# was not an OpenID identity page), the library will return
# an error code. Let the user know that that URL is unusable.
if status in [consumer.HTTP_FAILURE, consumer.PARSE_ERROR]:
if status == consumer.HTTP_FAILURE:
fmt = 'Failed to retrieve <q>%s</q>'
else:
fmt = 'Could not find OpenID information in <q>%s</q>'
message = fmt % (cgi.escape(openid_url),)
return self.render(request, message, css_class='error', form_contents=openid_url)
elif status == consumer.SUCCESS:
# The URL was a valid identity URL. Now we construct a URL
# that will get us to process the server response. We will
# need the token from the beginAuth call when processing
# the response. A cookie or a session object could be used
# to accomplish this, but for simplicity here we just add
# it as a query parameter of the return-to URL.
return_to = self.build_url(request, 'process', token=info.token)
# Now ask the library for the URL to redirect the user to
# his OpenID server. It is required for security that the
# return_to URL must be under the specified trust_root. We
# just use the base_url for this server as a trust root.
redirect_url = oidconsumer.constructRedirect(
info, return_to, trust_root=request['base_url'])
# Send the redirect response
return self.redirect(request, redirect_url)
else:
assert False, 'Not reached'
def do_process(self, request):
"""Handle the redirect from the OpenID server.
"""
oidconsumer = self.oidconsumer
# retrieve the token from the environment (in this case, the URL)
token = request['query'].get('token', '')
# Ask the library to check the response that the server sent
# us. Status is a code indicating the response type. info is
# either None or a string containing more information about
# the return type.
status, info = oidconsumer.completeAuth(token, request['query'])
css_class = 'error'
openid_url = None
if status == consumer.FAILURE and info:
# In the case of failure, if info is non-None, it is the
# URL that we were verifying. We include it in the error
# message to help the user figure out what happened.
openid_url = info
fmt = "Verification of %s failed."
message = fmt % (cgi.escape(openid_url),)
elif status == consumer.SUCCESS:
# Success means that the transaction completed without
# error. If info is None, it means that the user cancelled
# the verification.
css_class = 'alert'
if info:
# This is a successful verification attempt. If this
# was a real application, we would do our login,
# comment posting, etc. here.
openid_url = info
if self.url_to_username:
username = self.url_to_username(request['environ'], openid_url)
else:
username = openid_url
if 'paste.auth_tkt.set_user' in request['environ']:
request['environ']['paste.auth_tkt.set_user'](username)
if not self.login_redirect:
fmt = ("If you had supplied a login redirect path, you would have "
"been redirected there. "
"You have successfully verified %s as your identity.")
message = fmt % (cgi.escape(openid_url),)
else:
# @@: This stuff doesn't make sense to me; why not a remote redirect?
request['environ']['paste.auth.open_id'] = openid_url
request['environ']['PATH_INFO'] = self.login_redirect
return self.app(request['environ'], request['start'])
#exc = httpexceptions.HTTPTemporaryRedirect(self.login_redirect)
#return exc.wsgi_application(request['environ'], request['start'])
else:
# cancelled
message = 'Verification cancelled'
else:
# Either we don't understand the code or there is no
# openid_url included with the error. Give a generic
# failure message. The library should supply debug
# information in a log.
message = 'Verification failed.'
return self.render(request, message, css_class, openid_url)
def build_url(self, request, action, **query):
"""Build a URL relative to the server base_url, with the given
query parameters added."""
base = urlparse.urljoin(request['base_url'], self.auth_prefix + '/' + action)
return appendArgs(base, query)
def redirect(self, request, redirect_url):
"""Send a redirect response to the given URL to the browser."""
response_headers = [('Content-type', 'text/plain'),
('Location', redirect_url)]
request['start']('302 REDIRECT', response_headers)
return ["Redirecting to %s" % redirect_url]
def not_found(self, request):
"""Render a page with a 404 return code and a message."""
fmt = 'The path <q>%s</q> was not understood by this server.'
msg = fmt % (request['parsed_uri'],)
openid_url = request['query'].get('openid_url')
return self.render(request, msg, 'error', openid_url, status='404 Not Found')
def render(self, request, message=None, css_class='alert', form_contents=None,
status='200 OK', title="Python OpenID Consumer"):
"""Render a page."""
response_headers = [('Content-type', 'text/html')]
request['start'](str(status), response_headers)
self.page_header(request, title)
if message:
request['body'].append("<div class='%s'>" % (css_class,))
request['body'].append(message)
request['body'].append("</div>")
self.page_footer(request, form_contents)
return request['body']
def page_header(self, request, title):
"""Render the page header"""
request['body'].append('''\
<html>
<head><title>%s</title></head>
<style type="text/css">
* {
font-family: verdana,sans-serif;
}
body {
width: 50em;
margin: 1em;
}
div {
padding: .5em;
}
table {
margin: none;
padding: none;
}
.alert {
border: 1px solid #e7dc2b;
background: #fff888;
}
.error {
border: 1px solid #ff0000;
background: #ffaaaa;
}
#verify-form {
border: 1px solid #777777;
background: #dddddd;
margin-top: 1em;
padding-bottom: 0em;
}
</style>
<body>
<h1>%s</h1>
<p>
This example consumer uses the <a
href="http://openid.schtuff.com/">Python OpenID</a> library. It
just verifies that the URL that you enter is your identity URL.
</p>
''' % (title, title))
def page_footer(self, request, form_contents):
"""Render the page footer"""
if not form_contents:
form_contents = ''
request['body'].append('''\
<div id="verify-form">
<form method="get" action=%s>
Identity&nbsp;URL:
<input type="text" name="openid_url" value=%s />
<input type="submit" value="Verify" />
</form>
</div>
</body>
</html>
''' % (quoteattr(self.build_url(request, 'verify')), quoteattr(form_contents)))
middleware = AuthOpenIDHandler
def make_open_id_middleware(
app,
global_conf,
# Should this default to something, or inherit something from global_conf?:
data_store_path,
auth_prefix='/oid',
login_redirect=None,
catch_401=False,
url_to_username=None,
apply_auth_tkt=False,
auth_tkt_logout_path=None):
from paste.deploy.converters import asbool
from paste.util import import_string
catch_401 = asbool(catch_401)
if url_to_username and isinstance(url_to_username, basestring):
url_to_username = import_string.eval_import(url_to_username)
apply_auth_tkt = asbool(apply_auth_tkt)
new_app = AuthOpenIDHandler(
app, data_store_path=data_store_path, auth_prefix=auth_prefix,
login_redirect=login_redirect, catch_401=catch_401,
url_to_username=url_to_username or None)
if apply_auth_tkt:
from paste.auth import auth_tkt
new_app = auth_tkt.make_auth_tkt_middleware(
new_app, global_conf, logout_path=auth_tkt_logout_path)
return new_app

View File

@@ -0,0 +1,133 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
"""
Cascades through several applications, so long as applications
return ``404 Not Found``.
"""
from paste import httpexceptions
from paste.util import converters
import tempfile
from cStringIO import StringIO
__all__ = ['Cascade']
def make_cascade(loader, global_conf, catch='404', **local_conf):
"""
Entry point for Paste Deploy configuration
Expects configuration like::
[composit:cascade]
use = egg:Paste#cascade
# all start with 'app' and are sorted alphabetically
app1 = foo
app2 = bar
...
catch = 404 500 ...
"""
catch = map(int, converters.aslist(catch))
apps = []
for name, value in local_conf.items():
if not name.startswith('app'):
raise ValueError(
"Bad configuration key %r (=%r); all configuration keys "
"must start with 'app'"
% (name, value))
app = loader.get_app(value, global_conf=global_conf)
apps.append((name, app))
apps.sort()
apps = [app for name, app in apps]
return Cascade(apps, catch=catch)
class Cascade(object):
"""
Passed a list of applications, ``Cascade`` will try each of them
in turn. If one returns a status code listed in ``catch`` (by
default just ``404 Not Found``) then the next application is
tried.
If all applications fail, then the last application's failure
response is used.
Instances of this class are WSGI applications.
"""
def __init__(self, applications, catch=(404,)):
self.apps = applications
self.catch_codes = {}
self.catch_exceptions = []
for error in catch:
if isinstance(error, str):
error = int(error.split(None, 1)[0])
if isinstance(error, httpexceptions.HTTPException):
exc = error
code = error.code
else:
exc = httpexceptions.get_exception(error)
code = error
self.catch_codes[code] = exc
self.catch_exceptions.append(exc)
self.catch_exceptions = tuple(self.catch_exceptions)
def __call__(self, environ, start_response):
"""
WSGI application interface
"""
failed = []
def repl_start_response(status, headers, exc_info=None):
code = int(status.split(None, 1)[0])
if code in self.catch_codes:
failed.append(None)
return _consuming_writer
return start_response(status, headers, exc_info)
try:
length = int(environ.get('CONTENT_LENGTH', 0) or 0)
except ValueError:
length = 0
if length > 0:
# We have to copy wsgi.input
copy_wsgi_input = True
if length > 4096 or length < 0:
f = tempfile.TemporaryFile()
if length < 0:
f.write(environ['wsgi.input'].read())
else:
copy_len = length
while copy_len > 0:
chunk = environ['wsgi.input'].read(min(copy_len, 4096))
if not chunk:
raise IOError("Request body truncated")
f.write(chunk)
copy_len -= len(chunk)
f.seek(0)
else:
f = StringIO(environ['wsgi.input'].read(length))
environ['wsgi.input'] = f
else:
copy_wsgi_input = False
for app in self.apps[:-1]:
environ_copy = environ.copy()
if copy_wsgi_input:
environ_copy['wsgi.input'].seek(0)
failed = []
try:
v = app(environ_copy, repl_start_response)
if not failed:
return v
else:
if hasattr(v, 'close'):
# Exhaust the iterator first:
list(v)
# then close:
v.close()
except self.catch_exceptions, e:
pass
if copy_wsgi_input:
environ['wsgi.input'].seek(0)
return self.apps[-1](environ, start_response)
def _consuming_writer(s):
pass

View File

@@ -0,0 +1,276 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
"""
Application that runs a CGI script.
"""
import os
import sys
import subprocess
import urllib
try:
import select
except ImportError:
select = None
from paste.util import converters
__all__ = ['CGIError', 'CGIApplication']
class CGIError(Exception):
"""
Raised when the CGI script can't be found or doesn't
act like a proper CGI script.
"""
class CGIApplication(object):
"""
This object acts as a proxy to a CGI application. You pass in the
script path (``script``), an optional path to search for the
script (if the name isn't absolute) (``path``). If you don't give
a path, then ``$PATH`` will be used.
"""
def __init__(self,
global_conf,
script,
path=None,
include_os_environ=True,
query_string=None):
if global_conf:
raise NotImplemented(
"global_conf is no longer supported for CGIApplication "
"(use make_cgi_application); please pass None instead")
self.script_filename = script
if path is None:
path = os.environ.get('PATH', '').split(':')
self.path = path
if '?' in script:
assert query_string is None, (
"You cannot have '?' in your script name (%r) and also "
"give a query_string (%r)" % (script, query_string))
script, query_string = script.split('?', 1)
if os.path.abspath(script) != script:
# relative path
for path_dir in self.path:
if os.path.exists(os.path.join(path_dir, script)):
self.script = os.path.join(path_dir, script)
break
else:
raise CGIError(
"Script %r not found in path %r"
% (script, self.path))
else:
self.script = script
self.include_os_environ = include_os_environ
self.query_string = query_string
def __call__(self, environ, start_response):
if 'REQUEST_URI' not in environ:
environ['REQUEST_URI'] = (
urllib.quote(environ.get('SCRIPT_NAME', ''))
+ urllib.quote(environ.get('PATH_INFO', '')))
if self.include_os_environ:
cgi_environ = os.environ.copy()
else:
cgi_environ = {}
for name in environ:
# Should unicode values be encoded?
if (name.upper() == name
and isinstance(environ[name], str)):
cgi_environ[name] = environ[name]
if self.query_string is not None:
old = cgi_environ.get('QUERY_STRING', '')
if old:
old += '&'
cgi_environ['QUERY_STRING'] = old + self.query_string
cgi_environ['SCRIPT_FILENAME'] = self.script
proc = subprocess.Popen(
[self.script],
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
env=cgi_environ,
cwd=os.path.dirname(self.script),
)
writer = CGIWriter(environ, start_response)
if select and sys.platform != 'win32':
proc_communicate(
proc,
stdin=StdinReader.from_environ(environ),
stdout=writer,
stderr=environ['wsgi.errors'])
else:
stdout, stderr = proc.communicate(StdinReader.from_environ(environ).read())
if stderr:
environ['wsgi.errors'].write(stderr)
writer.write(stdout)
if not writer.headers_finished:
start_response(writer.status, writer.headers)
return []
class CGIWriter(object):
def __init__(self, environ, start_response):
self.environ = environ
self.start_response = start_response
self.status = '200 OK'
self.headers = []
self.headers_finished = False
self.writer = None
self.buffer = ''
def write(self, data):
if self.headers_finished:
self.writer(data)
return
self.buffer += data
while '\n' in self.buffer:
if '\r\n' in self.buffer and self.buffer.find('\r\n') < self.buffer.find('\n'):
line1, self.buffer = self.buffer.split('\r\n', 1)
else:
line1, self.buffer = self.buffer.split('\n', 1)
if not line1:
self.headers_finished = True
self.writer = self.start_response(
self.status, self.headers)
self.writer(self.buffer)
del self.buffer
del self.headers
del self.status
break
elif ':' not in line1:
raise CGIError(
"Bad header line: %r" % line1)
else:
name, value = line1.split(':', 1)
value = value.lstrip()
name = name.strip()
if name.lower() == 'status':
if ' ' not in value:
# WSGI requires this space, sometimes CGI scripts don't set it:
value = '%s General' % value
self.status = value
else:
self.headers.append((name, value))
class StdinReader(object):
def __init__(self, stdin, content_length):
self.stdin = stdin
self.content_length = content_length
def from_environ(cls, environ):
length = environ.get('CONTENT_LENGTH')
if length:
length = int(length)
else:
length = 0
return cls(environ['wsgi.input'], length)
from_environ = classmethod(from_environ)
def read(self, size=None):
if not self.content_length:
return ''
if size is None:
text = self.stdin.read(self.content_length)
else:
text = self.stdin.read(min(self.content_length, size))
self.content_length -= len(text)
return text
def proc_communicate(proc, stdin=None, stdout=None, stderr=None):
"""
Run the given process, piping input/output/errors to the given
file-like objects (which need not be actual file objects, unlike
the arguments passed to Popen). Wait for process to terminate.
Note: this is taken from the posix version of
subprocess.Popen.communicate, but made more general through the
use of file-like objects.
"""
read_set = []
write_set = []
input_buffer = ''
trans_nl = proc.universal_newlines and hasattr(open, 'newlines')
if proc.stdin:
# Flush stdio buffer. This might block, if the user has
# been writing to .stdin in an uncontrolled fashion.
proc.stdin.flush()
if input:
write_set.append(proc.stdin)
else:
proc.stdin.close()
else:
assert stdin is None
if proc.stdout:
read_set.append(proc.stdout)
else:
assert stdout is None
if proc.stderr:
read_set.append(proc.stderr)
else:
assert stderr is None
while read_set or write_set:
rlist, wlist, xlist = select.select(read_set, write_set, [])
if proc.stdin in wlist:
# When select has indicated that the file is writable,
# we can write up to PIPE_BUF bytes without risk
# blocking. POSIX defines PIPE_BUF >= 512
next, input_buffer = input_buffer, ''
next_len = 512-len(next)
if next_len:
next += stdin.read(next_len)
if not next:
proc.stdin.close()
write_set.remove(proc.stdin)
else:
bytes_written = os.write(proc.stdin.fileno(), next)
if bytes_written < len(next):
input_buffer = next[bytes_written:]
if proc.stdout in rlist:
data = os.read(proc.stdout.fileno(), 1024)
if data == "":
proc.stdout.close()
read_set.remove(proc.stdout)
if trans_nl:
data = proc._translate_newlines(data)
stdout.write(data)
if proc.stderr in rlist:
data = os.read(proc.stderr.fileno(), 1024)
if data == "":
proc.stderr.close()
read_set.remove(proc.stderr)
if trans_nl:
data = proc._translate_newlines(data)
stderr.write(data)
try:
proc.wait()
except OSError, e:
if e.errno != 10:
raise
def make_cgi_application(global_conf, script, path=None, include_os_environ=None,
query_string=None):
"""
Paste Deploy interface for :class:`CGIApplication`
This object acts as a proxy to a CGI application. You pass in the
script path (``script``), an optional path to search for the
script (if the name isn't absolute) (``path``). If you don't give
a path, then ``$PATH`` will be used.
"""
if path is None:
path = global_conf.get('path') or global_conf.get('PATH')
include_os_environ = converters.asbool(include_os_environ)
return CGIApplication(
script, path=path, include_os_environ=include_os_environ,
query_string=query_string)

View File

@@ -0,0 +1,116 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
"""
WSGI middleware
Captures any exceptions and prints a pretty report. See the `cgitb
documentation <http://python.org/doc/current/lib/module-cgitb.html>`_
for more.
"""
import cgitb
from cStringIO import StringIO
import sys
from paste.util import converters
class NoDefault(object):
pass
class CgitbMiddleware(object):
def __init__(self, app,
global_conf=None,
display=NoDefault,
logdir=None,
context=5,
format="html"):
self.app = app
if global_conf is None:
global_conf = {}
if display is NoDefault:
display = global_conf.get('debug')
if isinstance(display, basestring):
display = converters.asbool(display)
self.display = display
self.logdir = logdir
self.context = int(context)
self.format = format
def __call__(self, environ, start_response):
try:
app_iter = self.app(environ, start_response)
return self.catching_iter(app_iter, environ)
except:
exc_info = sys.exc_info()
start_response('500 Internal Server Error',
[('content-type', 'text/html')],
exc_info)
response = self.exception_handler(exc_info, environ)
return [response]
def catching_iter(self, app_iter, environ):
if not app_iter:
raise StopIteration
error_on_close = False
try:
for v in app_iter:
yield v
if hasattr(app_iter, 'close'):
error_on_close = True
app_iter.close()
except:
response = self.exception_handler(sys.exc_info(), environ)
if not error_on_close and hasattr(app_iter, 'close'):
try:
app_iter.close()
except:
close_response = self.exception_handler(
sys.exc_info(), environ)
response += (
'<hr noshade>Error in .close():<br>%s'
% close_response)
yield response
def exception_handler(self, exc_info, environ):
dummy_file = StringIO()
hook = cgitb.Hook(file=dummy_file,
display=self.display,
logdir=self.logdir,
context=self.context,
format=self.format)
hook(*exc_info)
return dummy_file.getvalue()
def make_cgitb_middleware(app, global_conf,
display=NoDefault,
logdir=None,
context=5,
format='html'):
"""
Wraps the application in the ``cgitb`` (standard library)
error catcher.
display:
If true (or debug is set in the global configuration)
then the traceback will be displayed in the browser
logdir:
Writes logs of all errors in that directory
context:
Number of lines of context to show around each line of
source code
"""
from paste.deploy.converters import asbool
if display is not NoDefault:
display = asbool(display)
if 'debug' in global_conf:
global_conf['debug'] = asbool(global_conf['debug'])
return CgitbMiddleware(
app, global_conf=global_conf,
display=display,
logdir=logdir,
context=context,
format=format)

View File

@@ -0,0 +1,120 @@
# (c) 2006 Ian Bicking, Philip Jenvey and contributors
# Written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
"""Paste Configuration Middleware and Objects"""
from paste.registry import RegistryManager, StackedObjectProxy
__all__ = ['DispatchingConfig', 'CONFIG', 'ConfigMiddleware']
class DispatchingConfig(StackedObjectProxy):
"""
This is a configuration object that can be used globally,
imported, have references held onto. The configuration may differ
by thread (or may not).
Specific configurations are registered (and deregistered) either
for the process or for threads.
"""
# @@: What should happen when someone tries to add this
# configuration to itself? Probably the conf should become
# resolved, and get rid of this delegation wrapper
def __init__(self, name='DispatchingConfig'):
super(DispatchingConfig, self).__init__(name=name)
self.__dict__['_process_configs'] = []
def push_thread_config(self, conf):
"""
Make ``conf`` the active configuration for this thread.
Thread-local configuration always overrides process-wide
configuration.
This should be used like::
conf = make_conf()
dispatching_config.push_thread_config(conf)
try:
... do stuff ...
finally:
dispatching_config.pop_thread_config(conf)
"""
self._push_object(conf)
def pop_thread_config(self, conf=None):
"""
Remove a thread-local configuration. If ``conf`` is given,
it is checked against the popped configuration and an error
is emitted if they don't match.
"""
self._pop_object(conf)
def push_process_config(self, conf):
"""
Like push_thread_config, but applies the configuration to
the entire process.
"""
self._process_configs.append(conf)
def pop_process_config(self, conf=None):
self._pop_from(self._process_configs, conf)
def _pop_from(self, lst, conf):
popped = lst.pop()
if conf is not None and popped is not conf:
raise AssertionError(
"The config popped (%s) is not the same as the config "
"expected (%s)"
% (popped, conf))
def _current_obj(self):
try:
return super(DispatchingConfig, self)._current_obj()
except TypeError:
if self._process_configs:
return self._process_configs[-1]
raise AttributeError(
"No configuration has been registered for this process "
"or thread")
current = current_conf = _current_obj
CONFIG = DispatchingConfig()
no_config = object()
class ConfigMiddleware(RegistryManager):
"""
A WSGI middleware that adds a ``paste.config`` key (by default)
to the request environment, as well as registering the
configuration temporarily (for the length of the request) with
``paste.config.CONFIG`` (or any other ``DispatchingConfig``
object).
"""
def __init__(self, application, config, dispatching_config=CONFIG,
environ_key='paste.config'):
"""
This delegates all requests to `application`, adding a *copy*
of the configuration `config`.
"""
def register_config(environ, start_response):
popped_config = environ.get(environ_key, no_config)
current_config = environ[environ_key] = config.copy()
environ['paste.registry'].register(dispatching_config,
current_config)
try:
app_iter = application(environ, start_response)
finally:
if popped_config is no_config:
environ.pop(environ_key, None)
else:
environ[environ_key] = popped_config
return app_iter
super(self.__class__, self).__init__(register_config)
def make_config_filter(app, global_conf, **local_conf):
conf = global_conf.copy()
conf.update(local_conf)
return ConfigMiddleware(app, conf)
make_config_middleware = ConfigMiddleware.__doc__

View File

@@ -0,0 +1,104 @@
# Cowbell images: http://commons.wikimedia.org/wiki/Image:Cowbell-1.jpg
import os
import re
from paste.fileapp import FileApp
from paste.response import header_value, remove_header
SOUND = "http://www.c-eye.net/eyeon/WalkenWAVS/explorestudiospace.wav"
class MoreCowbell(object):
def __init__(self, app):
self.app = app
def __call__(self, environ, start_response):
path_info = environ.get('PATH_INFO', '')
script_name = environ.get('SCRIPT_NAME', '')
for filename in ['bell-ascending.png', 'bell-descending.png']:
if path_info == '/.cowbell/'+ filename:
app = FileApp(os.path.join(os.path.dirname(__file__), filename))
return app(environ, start_response)
type = []
body = []
def repl_start_response(status, headers, exc_info=None):
ct = header_value(headers, 'content-type')
if ct and ct.startswith('text/html'):
type.append(ct)
remove_header(headers, 'content-length')
start_response(status, headers, exc_info)
return body.append
return start_response(status, headers, exc_info)
app_iter = self.app(environ, repl_start_response)
if type:
# Got text/html
body.extend(app_iter)
body = ''.join(body)
body = insert_head(body, self.javascript.replace('__SCRIPT_NAME__', script_name))
body = insert_body(body, self.resources.replace('__SCRIPT_NAME__', script_name))
return [body]
else:
return app_iter
javascript = '''\
<script type="text/javascript">
var cowbellState = 'hidden';
var lastCowbellPosition = null;
function showSomewhere() {
var sec, el;
if (cowbellState == 'hidden') {
el = document.getElementById('cowbell-ascending');
lastCowbellPosition = [parseInt(Math.random()*(window.innerWidth-200)),
parseInt(Math.random()*(window.innerHeight-200))];
el.style.left = lastCowbellPosition[0] + 'px';
el.style.top = lastCowbellPosition[1] + 'px';
el.style.display = '';
cowbellState = 'ascending';
sec = 1;
} else if (cowbellState == 'ascending') {
document.getElementById('cowbell-ascending').style.display = 'none';
el = document.getElementById('cowbell-descending');
el.style.left = lastCowbellPosition[0] + 'px';
el.style.top = lastCowbellPosition[1] + 'px';
el.style.display = '';
cowbellState = 'descending';
sec = 1;
} else {
document.getElementById('cowbell-descending').style.display = 'none';
cowbellState = 'hidden';
sec = Math.random()*20;
}
setTimeout(showSomewhere, sec*1000);
}
setTimeout(showSomewhere, Math.random()*20*1000);
</script>
'''
resources = '''\
<div id="cowbell-ascending" style="display: none; position: fixed">
<img src="__SCRIPT_NAME__/.cowbell/bell-ascending.png">
</div>
<div id="cowbell-descending" style="display: none; position: fixed">
<img src="__SCRIPT_NAME__/.cowbell/bell-descending.png">
</div>
'''
def insert_head(body, text):
end_head = re.search(r'</head>', body, re.I)
if end_head:
return body[:end_head.start()] + text + body[end_head.end():]
else:
return text + body
def insert_body(body, text):
end_body = re.search(r'</body>', body, re.I)
if end_body:
return body[:end_body.start()] + text + body[end_body.end():]
else:
return body + text
def make_cowbell(global_conf, app):
return MoreCowbell(app)
if __name__ == '__main__':
from paste.debug.debugapp import SimpleApplication
app = MoreCowbell(SimpleApplication())
from paste.httpserver import serve
serve(app)

View File

@@ -0,0 +1,5 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
"""
Package for debugging and development tools
"""

View File

@@ -0,0 +1,79 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
# (c) 2005 Clark C. Evans
# This module is part of the Python Paste Project and is released under
# the MIT License: http://www.opensource.org/licenses/mit-license.php
# This code was written with funding by http://prometheusresearch.com
"""
Various Applications for Debugging/Testing Purposes
"""
import time
__all__ = ['SimpleApplication', 'SlowConsumer']
class SimpleApplication(object):
"""
Produces a simple web page
"""
def __call__(self, environ, start_response):
body = "<html><body>simple</body></html>"
start_response("200 OK", [('Content-Type', 'text/html'),
('Content-Length', str(len(body)))])
return [body]
class SlowConsumer(object):
"""
Consumes an upload slowly...
NOTE: This should use the iterator form of ``wsgi.input``,
but it isn't implemented in paste.httpserver.
"""
def __init__(self, chunk_size = 4096, delay = 1, progress = True):
self.chunk_size = chunk_size
self.delay = delay
self.progress = True
def __call__(self, environ, start_response):
size = 0
total = environ.get('CONTENT_LENGTH')
if total:
remaining = int(total)
while remaining > 0:
if self.progress:
print "%s of %s remaining" % (remaining, total)
if remaining > 4096:
chunk = environ['wsgi.input'].read(4096)
else:
chunk = environ['wsgi.input'].read(remaining)
if not chunk:
break
size += len(chunk)
remaining -= len(chunk)
if self.delay:
time.sleep(self.delay)
body = "<html><body>%d bytes</body></html>" % size
else:
body = ('<html><body>\n'
'<form method="post" enctype="multipart/form-data">\n'
'<input type="file" name="file">\n'
'<input type="submit" >\n'
'</form></body></html>\n')
print "bingles"
start_response("200 OK", [('Content-Type', 'text/html'),
('Content-Length', len(body))])
return [body]
def make_test_app(global_conf):
return SimpleApplication()
make_test_app.__doc__ = SimpleApplication.__doc__
def make_slow_app(global_conf, chunk_size=4096, delay=1, progress=True):
from paste.deploy.converters import asbool
return SlowConsumer(
chunk_size=int(chunk_size),
delay=int(delay),
progress=asbool(progress))
make_slow_app.__doc__ = SlowConsumer.__doc__

View File

@@ -0,0 +1,435 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
#!/usr/bin/env python2.4
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
"""
These are functions for use when doctest-testing a document.
"""
try:
import subprocess
except ImportError:
from paste.util import subprocess24 as subprocess
import doctest
import os
import sys
import shutil
import re
import cgi
import rfc822
from cStringIO import StringIO
from paste.util import PySourceColor
here = os.path.abspath(__file__)
paste_parent = os.path.dirname(
os.path.dirname(os.path.dirname(here)))
def run(command):
data = run_raw(command)
if data:
print data
def run_raw(command):
"""
Runs the string command, returns any output.
"""
proc = subprocess.Popen(command, shell=True,
stderr=subprocess.STDOUT,
stdout=subprocess.PIPE, env=_make_env())
data = proc.stdout.read()
proc.wait()
while data.endswith('\n') or data.endswith('\r'):
data = data[:-1]
if data:
data = '\n'.join(
[l for l in data.splitlines() if l])
return data
else:
return ''
def run_command(command, name, and_print=False):
output = run_raw(command)
data = '$ %s\n%s' % (command, output)
show_file('shell-command', name, description='shell transcript',
data=data)
if and_print and output:
print output
def _make_env():
env = os.environ.copy()
env['PATH'] = (env.get('PATH', '')
+ ':'
+ os.path.join(paste_parent, 'scripts')
+ ':'
+ os.path.join(paste_parent, 'paste', '3rd-party',
'sqlobject-files', 'scripts'))
env['PYTHONPATH'] = (env.get('PYTHONPATH', '')
+ ':'
+ paste_parent)
return env
def clear_dir(dir):
"""
Clears (deletes) the given directory
"""
shutil.rmtree(dir, True)
def ls(dir=None, recurse=False, indent=0):
"""
Show a directory listing
"""
dir = dir or os.getcwd()
fns = os.listdir(dir)
fns.sort()
for fn in fns:
full = os.path.join(dir, fn)
if os.path.isdir(full):
fn = fn + '/'
print ' '*indent + fn
if os.path.isdir(full) and recurse:
ls(dir=full, recurse=True, indent=indent+2)
default_app = None
default_url = None
def set_default_app(app, url):
global default_app
global default_url
default_app = app
default_url = url
def resource_filename(fn):
"""
Returns the filename of the resource -- generally in the directory
resources/DocumentName/fn
"""
return os.path.join(
os.path.dirname(sys.testing_document_filename),
'resources',
os.path.splitext(os.path.basename(sys.testing_document_filename))[0],
fn)
def show(path_info, example_name):
fn = resource_filename(example_name + '.html')
out = StringIO()
assert default_app is not None, (
"No default_app set")
url = default_url + path_info
out.write('<span class="doctest-url"><a href="%s">%s</a></span><br>\n'
% (url, url))
out.write('<div class="doctest-example">\n')
proc = subprocess.Popen(
['paster', 'serve' '--server=console', '--no-verbose',
'--url=' + path_info],
stderr=subprocess.PIPE,
stdout=subprocess.PIPE,
env=_make_env())
stdout, errors = proc.communicate()
stdout = StringIO(stdout)
headers = rfc822.Message(stdout)
content = stdout.read()
for header, value in headers.items():
if header.lower() == 'status' and int(value.split()[0]) == 200:
continue
if header.lower() in ('content-type', 'content-length'):
continue
if (header.lower() == 'set-cookie'
and value.startswith('_SID_')):
continue
out.write('<span class="doctest-header">%s: %s</span><br>\n'
% (header, value))
lines = [l for l in content.splitlines() if l.strip()]
for line in lines:
out.write(line + '\n')
if errors:
out.write('<pre class="doctest-errors">%s</pre>'
% errors)
out.write('</div>\n')
result = out.getvalue()
if not os.path.exists(fn):
f = open(fn, 'wb')
f.write(result)
f.close()
else:
f = open(fn, 'rb')
expected = f.read()
f.close()
if not html_matches(expected, result):
print 'Pages did not match. Expected from %s:' % fn
print '-'*60
print expected
print '='*60
print 'Actual output:'
print '-'*60
print result
def html_matches(pattern, text):
regex = re.escape(pattern)
regex = regex.replace(r'\.\.\.', '.*')
regex = re.sub(r'0x[0-9a-f]+', '.*', regex)
regex = '^%s$' % regex
return re.search(regex, text)
def convert_docstring_string(data):
if data.startswith('\n'):
data = data[1:]
lines = data.splitlines()
new_lines = []
for line in lines:
if line.rstrip() == '.':
new_lines.append('')
else:
new_lines.append(line)
data = '\n'.join(new_lines) + '\n'
return data
def create_file(path, version, data):
data = convert_docstring_string(data)
write_data(path, data)
show_file(path, version)
def append_to_file(path, version, data):
data = convert_docstring_string(data)
f = open(path, 'a')
f.write(data)
f.close()
# I think these appends can happen so quickly (in less than a second)
# that the .pyc file doesn't appear to be expired, even though it
# is after we've made this change; so we have to get rid of the .pyc
# file:
if path.endswith('.py'):
pyc_file = path + 'c'
if os.path.exists(pyc_file):
os.unlink(pyc_file)
show_file(path, version, description='added to %s' % path,
data=data)
def show_file(path, version, description=None, data=None):
ext = os.path.splitext(path)[1]
if data is None:
f = open(path, 'rb')
data = f.read()
f.close()
if ext == '.py':
html = ('<div class="source-code">%s</div>'
% PySourceColor.str2html(data, PySourceColor.dark))
else:
html = '<pre class="source-code">%s</pre>' % cgi.escape(data, 1)
html = '<span class="source-filename">%s</span><br>%s' % (
description or path, html)
write_data(resource_filename('%s.%s.gen.html' % (path, version)),
html)
def call_source_highlight(input, format):
proc = subprocess.Popen(['source-highlight', '--out-format=html',
'--no-doc', '--css=none',
'--src-lang=%s' % format], shell=False,
stdout=subprocess.PIPE)
stdout, stderr = proc.communicate(input)
result = stdout
proc.wait()
return result
def write_data(path, data):
dir = os.path.dirname(os.path.abspath(path))
if not os.path.exists(dir):
os.makedirs(dir)
f = open(path, 'wb')
f.write(data)
f.close()
def change_file(path, changes):
f = open(os.path.abspath(path), 'rb')
lines = f.readlines()
f.close()
for change_type, line, text in changes:
if change_type == 'insert':
lines[line:line] = [text]
elif change_type == 'delete':
lines[line:text] = []
else:
assert 0, (
"Unknown change_type: %r" % change_type)
f = open(path, 'wb')
f.write(''.join(lines))
f.close()
class LongFormDocTestParser(doctest.DocTestParser):
"""
This parser recognizes some reST comments as commands, without
prompts or expected output, like:
.. run:
do_this(...
...)
"""
_EXAMPLE_RE = re.compile(r"""
# Source consists of a PS1 line followed by zero or more PS2 lines.
(?: (?P<source>
(?:^(?P<indent> [ ]*) >>> .*) # PS1 line
(?:\n [ ]* \.\.\. .*)*) # PS2 lines
\n?
# Want consists of any non-blank lines that do not start with PS1.
(?P<want> (?:(?![ ]*$) # Not a blank line
(?![ ]*>>>) # Not a line starting with PS1
.*$\n? # But any other line
)*))
|
(?: # This is for longer commands that are prefixed with a reST
# comment like '.. run:' (two colons makes that a directive).
# These commands cannot have any output.
(?:^\.\.[ ]*(?P<run>run):[ ]*\n) # Leading command/command
(?:[ ]*\n)? # Blank line following
(?P<runsource>
(?:(?P<runindent> [ ]+)[^ ].*$)
(?:\n [ ]+ .*)*)
)
|
(?: # This is for shell commands
(?P<shellsource>
(?:^(P<shellindent> [ ]*) [$] .*) # Shell line
(?:\n [ ]* [>] .*)*) # Continuation
\n?
# Want consists of any non-blank lines that do not start with $
(?P<shellwant> (?:(?![ ]*$)
(?![ ]*[$]$)
.*$\n?
)*))
""", re.MULTILINE | re.VERBOSE)
def _parse_example(self, m, name, lineno):
r"""
Given a regular expression match from `_EXAMPLE_RE` (`m`),
return a pair `(source, want)`, where `source` is the matched
example's source code (with prompts and indentation stripped);
and `want` is the example's expected output (with indentation
stripped).
`name` is the string's name, and `lineno` is the line number
where the example starts; both are used for error messages.
>>> def parseit(s):
... p = LongFormDocTestParser()
... return p._parse_example(p._EXAMPLE_RE.search(s), '<string>', 1)
>>> parseit('>>> 1\n1')
('1', {}, '1', None)
>>> parseit('>>> (1\n... +1)\n2')
('(1\n+1)', {}, '2', None)
>>> parseit('.. run:\n\n test1\n test2\n')
('test1\ntest2', {}, '', None)
"""
# Get the example's indentation level.
runner = m.group('run') or ''
indent = len(m.group('%sindent' % runner))
# Divide source into lines; check that they're properly
# indented; and then strip their indentation & prompts.
source_lines = m.group('%ssource' % runner).split('\n')
if runner:
self._check_prefix(source_lines[1:], ' '*indent, name, lineno)
else:
self._check_prompt_blank(source_lines, indent, name, lineno)
self._check_prefix(source_lines[2:], ' '*indent + '.', name, lineno)
if runner:
source = '\n'.join([sl[indent:] for sl in source_lines])
else:
source = '\n'.join([sl[indent+4:] for sl in source_lines])
if runner:
want = ''
exc_msg = None
else:
# Divide want into lines; check that it's properly indented; and
# then strip the indentation. Spaces before the last newline should
# be preserved, so plain rstrip() isn't good enough.
want = m.group('want')
want_lines = want.split('\n')
if len(want_lines) > 1 and re.match(r' *$', want_lines[-1]):
del want_lines[-1] # forget final newline & spaces after it
self._check_prefix(want_lines, ' '*indent, name,
lineno + len(source_lines))
want = '\n'.join([wl[indent:] for wl in want_lines])
# If `want` contains a traceback message, then extract it.
m = self._EXCEPTION_RE.match(want)
if m:
exc_msg = m.group('msg')
else:
exc_msg = None
# Extract options from the source.
options = self._find_options(source, name, lineno)
return source, options, want, exc_msg
def parse(self, string, name='<string>'):
"""
Divide the given string into examples and intervening text,
and return them as a list of alternating Examples and strings.
Line numbers for the Examples are 0-based. The optional
argument `name` is a name identifying this string, and is only
used for error messages.
"""
string = string.expandtabs()
# If all lines begin with the same indentation, then strip it.
min_indent = self._min_indent(string)
if min_indent > 0:
string = '\n'.join([l[min_indent:] for l in string.split('\n')])
output = []
charno, lineno = 0, 0
# Find all doctest examples in the string:
for m in self._EXAMPLE_RE.finditer(string):
# Add the pre-example text to `output`.
output.append(string[charno:m.start()])
# Update lineno (lines before this example)
lineno += string.count('\n', charno, m.start())
# Extract info from the regexp match.
(source, options, want, exc_msg) = \
self._parse_example(m, name, lineno)
# Create an Example, and add it to the list.
if not self._IS_BLANK_OR_COMMENT(source):
# @@: Erg, this is the only line I need to change...
output.append(doctest.Example(
source, want, exc_msg,
lineno=lineno,
indent=min_indent+len(m.group('indent') or m.group('runindent')),
options=options))
# Update lineno (lines inside this example)
lineno += string.count('\n', m.start(), m.end())
# Update charno.
charno = m.end()
# Add any remaining post-example text to `output`.
output.append(string[charno:])
return output
if __name__ == '__main__':
if sys.argv[1:] and sys.argv[1] == 'doctest':
doctest.testmod()
sys.exit()
if not paste_parent in sys.path:
sys.path.append(paste_parent)
for fn in sys.argv[1:]:
fn = os.path.abspath(fn)
# @@: OK, ick; but this module gets loaded twice
sys.testing_document_filename = fn
doctest.testfile(
fn, module_relative=False,
optionflags=doctest.ELLIPSIS|doctest.REPORT_ONLY_FIRST_FAILURE,
parser=LongFormDocTestParser())
new = os.path.splitext(fn)[0] + '.html'
assert new != fn
os.system('rst2html.py %s > %s' % (fn, new))

View File

@@ -0,0 +1,409 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
"""
Module to find differences over time in a filesystem
Basically this takes a snapshot of a directory, then sees what changes
were made. The contents of the files are not checked, so you can
detect that the content was changed, but not what the old version of
the file was.
"""
import os
from fnmatch import fnmatch
from datetime import datetime
from paste.util.UserDict24 import IterableUserDict
import operator
import re
__all__ = ['Diff', 'Snapshot', 'File', 'Dir', 'report_expected_diffs',
'show_diff']
class Diff(object):
"""
Represents the difference between two snapshots
"""
def __init__(self, before, after):
self.before = before
self.after = after
self._calculate()
def _calculate(self):
before = self.before.data
after = self.after.data
self.deleted = {}
self.updated = {}
self.created = after.copy()
for path, f in before.items():
if path not in after:
self.deleted[path] = f
continue
del self.created[path]
if f.mtime < after[path].mtime:
self.updated[path] = after[path]
def __str__(self):
return self.report()
def report(self, header=True, dates=False):
s = []
if header:
s.append('Difference in %s from %s to %s:' %
(self.before.base_path,
self.before.calculated,
self.after.calculated))
for name, files, show_size in [
('created', self.created, True),
('deleted', self.deleted, True),
('updated', self.updated, True)]:
if files:
s.append('-- %s: -------------------' % name)
files = files.items()
files.sort()
last = ''
for path, f in files:
t = ' %s' % _space_prefix(last, path, indent=4,
include_sep=False)
last = path
if show_size and f.size != 'N/A':
t += ' (%s bytes)' % f.size
if dates:
parts = []
if self.before.get(path):
parts.append(self.before[path].mtime)
if self.after.get(path):
parts.append(self.after[path].mtime)
t += ' (mtime: %s)' % ('->'.join(map(repr, parts)))
s.append(t)
if len(s) == 1:
s.append(' (no changes)')
return '\n'.join(s)
class Snapshot(IterableUserDict):
"""
Represents a snapshot of a set of files. Has a dictionary-like
interface, keyed relative to ``base_path``
"""
def __init__(self, base_path, files=None, ignore_wildcards=(),
ignore_paths=(), ignore_hidden=True):
self.base_path = base_path
self.ignore_wildcards = ignore_wildcards
self.ignore_hidden = ignore_hidden
self.ignore_paths = ignore_paths
self.calculated = None
self.data = files or {}
if files is None:
self.find_files()
############################################################
## File finding
############################################################
def find_files(self):
"""
Find all the files under the base path, and put them in
``self.data``
"""
self._find_traverse('', self.data)
self.calculated = datetime.now()
def _ignore_file(self, fn):
if fn in self.ignore_paths:
return True
if self.ignore_hidden and os.path.basename(fn).startswith('.'):
return True
for pat in self.ignore_wildcards:
if fnmatch(fn, pat):
return True
return False
def _ignore_file(self, fn):
if fn in self.ignore_paths:
return True
if self.ignore_hidden and os.path.basename(fn).startswith('.'):
return True
return False
def _find_traverse(self, path, result):
full = os.path.join(self.base_path, path)
if os.path.isdir(full):
if path:
# Don't actually include the base path
result[path] = Dir(self.base_path, path)
for fn in os.listdir(full):
fn = os.path.join(path, fn)
if self._ignore_file(fn):
continue
self._find_traverse(fn, result)
else:
result[path] = File(self.base_path, path)
def __repr__(self):
return '<%s in %r from %r>' % (
self.__class__.__name__, self.base_path,
self.calculated or '(no calculation done)')
def compare_expected(self, expected, comparison=operator.eq,
differ=None, not_found=None,
include_success=False):
"""
Compares a dictionary of ``path: content`` to the
found files. Comparison is done by equality, or the
``comparison(actual_content, expected_content)`` function given.
Returns dictionary of differences, keyed by path. Each
difference is either noted, or the output of
``differ(actual_content, expected_content)`` is given.
If a file does not exist and ``not_found`` is given, then
``not_found(path)`` is put in.
"""
result = {}
for path in expected:
orig_path = path
path = path.strip('/')
if path not in self.data:
if not_found:
msg = not_found(path)
else:
msg = 'not found'
result[path] = msg
continue
expected_content = expected[orig_path]
file = self.data[path]
actual_content = file.bytes
if not comparison(actual_content, expected_content):
if differ:
msg = differ(actual_content, expected_content)
else:
if len(actual_content) < len(expected_content):
msg = 'differ (%i bytes smaller)' % (
len(expected_content) - len(actual_content))
elif len(actual_content) > len(expected_content):
msg = 'differ (%i bytes larger)' % (
len(actual_content) - len(expected_content))
else:
msg = 'diff (same size)'
result[path] = msg
elif include_success:
result[path] = 'same!'
return result
def diff_to_now(self):
return Diff(self, self.clone())
def clone(self):
return self.__class__(base_path=self.base_path,
ignore_wildcards=self.ignore_wildcards,
ignore_paths=self.ignore_paths,
ignore_hidden=self.ignore_hidden)
class File(object):
"""
Represents a single file found as the result of a command.
Has attributes:
``path``:
The path of the file, relative to the ``base_path``
``full``:
The full path
``stat``:
The results of ``os.stat``. Also ``mtime`` and ``size``
contain the ``.st_mtime`` and ``st_size`` of the stat.
``bytes``:
The contents of the file.
You may use the ``in`` operator with these objects (tested against
the contents of the file), and the ``.mustcontain()`` method.
"""
file = True
dir = False
def __init__(self, base_path, path):
self.base_path = base_path
self.path = path
self.full = os.path.join(base_path, path)
self.stat = os.stat(self.full)
self.mtime = self.stat.st_mtime
self.size = self.stat.st_size
self._bytes = None
def bytes__get(self):
if self._bytes is None:
f = open(self.full, 'rb')
self._bytes = f.read()
f.close()
return self._bytes
bytes = property(bytes__get)
def __contains__(self, s):
return s in self.bytes
def mustcontain(self, s):
__tracebackhide__ = True
bytes = self.bytes
if s not in bytes:
print 'Could not find %r in:' % s
print bytes
assert s in bytes
def __repr__(self):
return '<%s %s:%s>' % (
self.__class__.__name__,
self.base_path, self.path)
class Dir(File):
"""
Represents a directory created by a command.
"""
file = False
dir = True
def __init__(self, base_path, path):
self.base_path = base_path
self.path = path
self.full = os.path.join(base_path, path)
self.size = 'N/A'
self.mtime = 'N/A'
def __repr__(self):
return '<%s %s:%s>' % (
self.__class__.__name__,
self.base_path, self.path)
def bytes__get(self):
raise NotImplementedError(
"Directory %r doesn't have content" % self)
bytes = property(bytes__get)
def _space_prefix(pref, full, sep=None, indent=None, include_sep=True):
"""
Anything shared by pref and full will be replaced with spaces
in full, and full returned.
Example::
>>> _space_prefix('/foo/bar', '/foo')
' /bar'
"""
if sep is None:
sep = os.path.sep
pref = pref.split(sep)
full = full.split(sep)
padding = []
while pref and full and pref[0] == full[0]:
if indent is None:
padding.append(' ' * (len(full[0]) + len(sep)))
else:
padding.append(' ' * indent)
full.pop(0)
pref.pop(0)
if padding:
if include_sep:
return ''.join(padding) + sep + sep.join(full)
else:
return ''.join(padding) + sep.join(full)
else:
return sep.join(full)
def report_expected_diffs(diffs, colorize=False):
"""
Takes the output of compare_expected, and returns a string
description of the differences.
"""
if not diffs:
return 'No differences'
diffs = diffs.items()
diffs.sort()
s = []
last = ''
for path, desc in diffs:
t = _space_prefix(last, path, indent=4, include_sep=False)
if colorize:
t = color_line(t, 11)
last = path
if len(desc.splitlines()) > 1:
cur_indent = len(re.search(r'^[ ]*', t).group(0))
desc = indent(cur_indent+2, desc)
if colorize:
t += '\n'
for line in desc.splitlines():
if line.strip().startswith('+'):
line = color_line(line, 10)
elif line.strip().startswith('-'):
line = color_line(line, 9)
else:
line = color_line(line, 14)
t += line+'\n'
else:
t += '\n' + desc
else:
t += ' '+desc
s.append(t)
s.append('Files with differences: %s' % len(diffs))
return '\n'.join(s)
def color_code(foreground=None, background=None):
"""
0 black
1 red
2 green
3 yellow
4 blue
5 magenta (purple)
6 cyan
7 white (gray)
Add 8 to get high-intensity
"""
if foreground is None and background is None:
# Reset
return '\x1b[0m'
codes = []
if foreground is None:
codes.append('[39m')
elif foreground > 7:
codes.append('[1m')
codes.append('[%im' % (22+foreground))
else:
codes.append('[%im' % (30+foreground))
if background is None:
codes.append('[49m')
else:
codes.append('[%im' % (40+background))
return '\x1b' + '\x1b'.join(codes)
def color_line(line, foreground=None, background=None):
match = re.search(r'^(\s*)', line)
return (match.group(1) + color_code(foreground, background)
+ line[match.end():] + color_code())
def indent(indent, text):
return '\n'.join(
[' '*indent + l for l in text.splitlines()])
def show_diff(actual_content, expected_content):
actual_lines = [l.strip() for l in actual_content.splitlines()
if l.strip()]
expected_lines = [l.strip() for l in expected_content.splitlines()
if l.strip()]
if len(actual_lines) == len(expected_lines) == 1:
return '%r not %r' % (actual_lines[0], expected_lines[0])
if not actual_lines:
return 'Empty; should have:\n'+expected_content
import difflib
return '\n'.join(difflib.ndiff(actual_lines, expected_lines))

View File

@@ -0,0 +1,148 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
"""
Middleware that displays everything that is printed inline in
application pages.
Anything printed during the request will get captured and included on
the page. It will usually be included as a floating element in the
top right hand corner of the page. If you want to override this
you can include a tag in your template where it will be placed::
<pre id="paste-debug-prints"></pre>
You might want to include ``style="white-space: normal"``, as all the
whitespace will be quoted, and this allows the text to wrap if
necessary.
"""
from cStringIO import StringIO
import re
import cgi
from paste.util import threadedprint
from paste import wsgilib
from paste import response
import sys
_threadedprint_installed = False
__all__ = ['PrintDebugMiddleware']
class TeeFile(object):
def __init__(self, files):
self.files = files
def write(self, v):
if isinstance(v, unicode):
# WSGI is picky in this case
v = str(v)
for file in self.files:
file.write(v)
class PrintDebugMiddleware(object):
"""
This middleware captures all the printed statements, and inlines
them in HTML pages, so that you can see all the (debug-intended)
print statements in the page itself.
There are two keys added to the environment to control this:
``environ['paste.printdebug_listeners']`` is a list of functions
that will be called everytime something is printed.
``environ['paste.remove_printdebug']`` is a function that, if
called, will disable printing of output for that request.
If you have ``replace_stdout=True`` then stdout is replaced, not
captured.
"""
log_template = (
'<pre style="width: 40%%; border: 2px solid #000; white-space: normal; '
'background-color: #ffd; color: #000; float: right;">'
'<b style="border-bottom: 1px solid #000">Log messages</b><br>'
'%s</pre>')
def __init__(self, app, global_conf=None, force_content_type=False,
print_wsgi_errors=True, replace_stdout=False):
# @@: global_conf should be handled separately and only for
# the entry point
self.app = app
self.force_content_type = force_content_type
if isinstance(print_wsgi_errors, basestring):
from paste.deploy.converters import asbool
print_wsgi_errors = asbool(print_wsgi_errors)
self.print_wsgi_errors = print_wsgi_errors
self.replace_stdout = replace_stdout
self._threaded_print_stdout = None
def __call__(self, environ, start_response):
global _threadedprint_installed
if environ.get('paste.testing'):
# In a testing environment this interception isn't
# useful:
return self.app(environ, start_response)
if (not _threadedprint_installed
or self._threaded_print_stdout is not sys.stdout):
# @@: Not strictly threadsafe
_threadedprint_installed = True
threadedprint.install(leave_stdout=not self.replace_stdout)
self._threaded_print_stdout = sys.stdout
removed = []
def remove_printdebug():
removed.append(None)
environ['paste.remove_printdebug'] = remove_printdebug
logged = StringIO()
listeners = [logged]
environ['paste.printdebug_listeners'] = listeners
if self.print_wsgi_errors:
listeners.append(environ['wsgi.errors'])
replacement_stdout = TeeFile(listeners)
threadedprint.register(replacement_stdout)
try:
status, headers, body = wsgilib.intercept_output(
environ, self.app)
if status is None:
# Some error occurred
status = '500 Server Error'
headers = [('Content-type', 'text/html')]
start_response(status, headers)
if not body:
body = 'An error occurred'
content_type = response.header_value(headers, 'content-type')
if (removed or
(not self.force_content_type and
(not content_type
or not content_type.startswith('text/html')))):
if replacement_stdout == logged:
# Then the prints will be lost, unless...
environ['wsgi.errors'].write(logged.getvalue())
start_response(status, headers)
return [body]
response.remove_header(headers, 'content-length')
body = self.add_log(body, logged.getvalue())
start_response(status, headers)
return [body]
finally:
threadedprint.deregister()
_body_re = re.compile(r'<body[^>]*>', re.I)
_explicit_re = re.compile(r'<pre\s*[^>]*id="paste-debug-prints".*?>',
re.I+re.S)
def add_log(self, html, log):
if not log:
return html
text = cgi.escape(log)
text = text.replace('\n', '<br>')
text = text.replace(' ', '&nbsp; ')
match = self._explicit_re.search(html)
if not match:
text = self.log_template % text
match = self._body_re.search(html)
if not match:
return text + html
else:
return html[:match.end()] + text + html[match.end():]

View File

@@ -0,0 +1,227 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
"""
Middleware that profiles the request and displays profiling
information at the bottom of each page.
"""
import sys
import os
import hotshot
import hotshot.stats
import threading
import cgi
import time
from cStringIO import StringIO
from paste import response
__all__ = ['ProfileMiddleware', 'profile_decorator']
class ProfileMiddleware(object):
"""
Middleware that profiles all requests.
All HTML pages will have profiling information appended to them.
The data is isolated to that single request, and does not include
data from previous requests.
This uses the ``hotshot`` module, which affects performance of the
application. It also runs in a single-threaded mode, so it is
only usable in development environments.
"""
style = ('clear: both; background-color: #ff9; color: #000; '
'border: 2px solid #000; padding: 5px;')
def __init__(self, app, global_conf=None,
log_filename='profile.log.tmp',
limit=40):
self.app = app
self.lock = threading.Lock()
self.log_filename = log_filename
self.limit = limit
def __call__(self, environ, start_response):
catch_response = []
body = []
def replace_start_response(status, headers, exc_info=None):
catch_response.extend([status, headers])
start_response(status, headers, exc_info)
return body.append
def run_app():
app_iter = self.app(environ, replace_start_response)
try:
body.extend(app_iter)
finally:
if hasattr(app_iter, 'close'):
app_iter.close()
self.lock.acquire()
try:
prof = hotshot.Profile(self.log_filename)
prof.addinfo('URL', environ.get('PATH_INFO', ''))
try:
prof.runcall(run_app)
finally:
prof.close()
body = ''.join(body)
headers = catch_response[1]
content_type = response.header_value(headers, 'content-type')
if content_type is None or not content_type.startswith('text/html'):
# We can't add info to non-HTML output
return [body]
stats = hotshot.stats.load(self.log_filename)
stats.strip_dirs()
stats.sort_stats('time', 'calls')
output = capture_output(stats.print_stats, self.limit)
output_callers = capture_output(
stats.print_callers, self.limit)
body += '<pre style="%s">%s\n%s</pre>' % (
self.style, cgi.escape(output), cgi.escape(output_callers))
return [body]
finally:
self.lock.release()
def capture_output(func, *args, **kw):
# Not threadsafe! (that's okay when ProfileMiddleware uses it,
# though, since it synchronizes itself.)
out = StringIO()
old_stdout = sys.stdout
sys.stdout = out
try:
func(*args, **kw)
finally:
sys.stdout = old_stdout
return out.getvalue()
def profile_decorator(**options):
"""
Profile a single function call.
Used around a function, like::
@profile_decorator(options...)
def ...
All calls to the function will be profiled. The options are
all keywords, and are:
log_file:
The filename to log to (or ``'stdout'`` or ``'stderr'``).
Default: stderr.
display_limit:
Only show the top N items, default: 20.
sort_stats:
A list of string-attributes to sort on. Default
``('time', 'calls')``.
strip_dirs:
Strip directories/module names from files? Default True.
add_info:
If given, this info will be added to the report (for your
own tracking). Default: none.
log_filename:
The temporary filename to log profiling data to. Default;
``./profile_data.log.tmp``
no_profile:
If true, then don't actually profile anything. Useful for
conditional profiling.
"""
if options.get('no_profile'):
def decorator(func):
return func
return decorator
def decorator(func):
def replacement(*args, **kw):
return DecoratedProfile(func, **options)(*args, **kw)
return replacement
return decorator
class DecoratedProfile(object):
lock = threading.Lock()
def __init__(self, func, **options):
self.func = func
self.options = options
def __call__(self, *args, **kw):
self.lock.acquire()
try:
return self.profile(self.func, *args, **kw)
finally:
self.lock.release()
def profile(self, func, *args, **kw):
ops = self.options
prof_filename = ops.get('log_filename', 'profile_data.log.tmp')
prof = hotshot.Profile(prof_filename)
prof.addinfo('Function Call',
self.format_function(func, *args, **kw))
if ops.get('add_info'):
prof.addinfo('Extra info', ops['add_info'])
exc_info = None
try:
start_time = time.time()
try:
result = prof.runcall(func, *args, **kw)
except:
exc_info = sys.exc_info()
end_time = time.time()
finally:
prof.close()
stats = hotshot.stats.load(prof_filename)
os.unlink(prof_filename)
if ops.get('strip_dirs', True):
stats.strip_dirs()
stats.sort_stats(*ops.get('sort_stats', ('time', 'calls')))
display_limit = ops.get('display_limit', 20)
output = capture_output(stats.print_stats, display_limit)
output_callers = capture_output(
stats.print_callers, display_limit)
output_file = ops.get('log_file')
if output_file in (None, 'stderr'):
f = sys.stderr
elif output_file in ('-', 'stdout'):
f = sys.stdout
else:
f = open(output_file, 'a')
f.write('\n%s\n' % ('-'*60))
f.write('Date: %s\n' % time.strftime('%c'))
f.write('Function call: %s\n'
% self.format_function(func, *args, **kw))
f.write('Wall time: %0.2f seconds\n'
% (end_time - start_time))
f.write(output)
f.write(output_callers)
if output_file not in (None, '-', 'stdout', 'stderr'):
f.close()
if exc_info:
# We captured an exception earlier, now we re-raise it
raise exc_info[0], exc_info[1], exc_info[2]
return result
def format_function(self, func, *args, **kw):
args = map(repr, args)
args.extend(
['%s=%r' % (k, v) for k, v in kw.items()])
return '%s(%s)' % (func.__name__, ', '.join(args))
def make_profile_middleware(
app, global_conf,
log_filename='profile.log.tmp',
limit=40):
"""
Wrap the application in a component that will profile each
request. The profiling data is then appended to the output
of each page.
Note that this serializes all requests (i.e., removing
concurrency). Therefore never use this in production.
"""
limit = int(limit)
return ProfileMiddleware(
app, log_filename=log_filename, limit=limit)

View File

@@ -0,0 +1,93 @@
# (c) 2005 Clark C. Evans
# This module is part of the Python Paste Project and is released under
# the MIT License: http://www.opensource.org/licenses/mit-license.php
# This code was written with funding by http://prometheusresearch.com
"""
WSGI Test Server
This builds upon paste.util.baseserver to customize it for regressions
where using raw_interactive won't do.
"""
import time
from paste.httpserver import *
class WSGIRegressionServer(WSGIServer):
"""
A threaded WSGIServer for use in regression testing. To use this
module, call serve(application, regression=True), and then call
server.accept() to let it handle one request. When finished, use
server.stop() to shutdown the server. Note that all pending requests
are processed before the server shuts down.
"""
defaulttimeout = 10
def __init__ (self, *args, **kwargs):
WSGIServer.__init__(self, *args, **kwargs)
self.stopping = []
self.pending = []
self.timeout = self.defaulttimeout
# this is a local connection, be quick
self.socket.settimeout(2)
def serve_forever(self):
from threading import Thread
thread = Thread(target=self.serve_pending)
thread.start()
def reset_expires(self):
if self.timeout:
self.expires = time.time() + self.timeout
def close_request(self, *args, **kwargs):
WSGIServer.close_request(self, *args, **kwargs)
self.pending.pop()
self.reset_expires()
def serve_pending(self):
self.reset_expires()
while not self.stopping or self.pending:
now = time.time()
if now > self.expires and self.timeout:
# note regression test doesn't handle exceptions in
# threads very well; so we just print and exit
print "\nWARNING: WSGIRegressionServer timeout exceeded\n"
break
if self.pending:
self.handle_request()
time.sleep(.1)
def stop(self):
""" stop the server (called from tester's thread) """
self.stopping.append(True)
def accept(self, count = 1):
""" accept another request (called from tester's thread) """
assert not self.stopping
[self.pending.append(True) for x in range(count)]
def serve(application, host=None, port=None, handler=None):
server = WSGIRegressionServer(application, host, port, handler)
print "serving on %s:%s" % server.server_address
server.serve_forever()
return server
if __name__ == '__main__':
import urllib
from paste.wsgilib import dump_environ
server = serve(dump_environ)
baseuri = ("http://%s:%s" % server.server_address)
def fetch(path):
# tell the server to humor exactly one more request
server.accept(1)
# not needed; but this is what you do if the server
# may not respond in a resonable time period
import socket
socket.setdefaulttimeout(5)
# build a uri, fetch and return
return urllib.urlopen(baseuri + path).read()
assert "PATH_INFO: /foo" in fetch("/foo")
assert "PATH_INFO: /womble" in fetch("/womble")
# ok, let's make one more final request...
server.accept(1)
# and then schedule a stop()
server.stop()
# and then... fetch it...
urllib.urlopen(baseuri)

View File

@@ -0,0 +1,347 @@
"""
Watches the key ``paste.httpserver.thread_pool`` to see how many
threads there are and report on any wedged threads.
"""
import sys
import cgi
import time
import traceback
from cStringIO import StringIO
from thread import get_ident
from paste import httpexceptions
from paste.request import construct_url, parse_formvars
from paste.util.template import HTMLTemplate, bunch
page_template = HTMLTemplate('''
<html>
<head>
<style type="text/css">
body {
font-family: sans-serif;
}
table.environ tr td {
border-bottom: #bbb 1px solid;
}
table.environ tr td.bottom {
border-bottom: none;
}
table.thread {
border: 1px solid #000;
margin-bottom: 1em;
}
table.thread tr td {
border-bottom: #999 1px solid;
padding-right: 1em;
}
table.thread tr td.bottom {
border-bottom: none;
}
table.thread tr.this_thread td {
background-color: #006;
color: #fff;
}
a.button {
background-color: #ddd;
border: #aaa outset 2px;
text-decoration: none;
margin-top: 10px;
font-size: 80%;
color: #000;
}
a.button:hover {
background-color: #eee;
border: #bbb outset 2px;
}
a.button:active {
border: #bbb inset 2px;
}
</style>
<title>{{title}}</title>
</head>
<body>
<h1>{{title}}</h1>
{{if kill_thread_id}}
<div style="background-color: #060; color: #fff;
border: 2px solid #000;">
Thread {{kill_thread_id}} killed
</div>
{{endif}}
<div>Pool size: {{nworkers}}
{{if actual_workers > nworkers}}
+ {{actual_workers-nworkers}} extra
{{endif}}
({{nworkers_used}} used including current request)<br>
idle: {{len(track_threads["idle"])}},
busy: {{len(track_threads["busy"])}},
hung: {{len(track_threads["hung"])}},
dying: {{len(track_threads["dying"])}},
zombie: {{len(track_threads["zombie"])}}</div>
{{for thread in threads}}
<table class="thread">
<tr {{if thread.thread_id == this_thread_id}}class="this_thread"{{endif}}>
<td>
<b>Thread</b>
{{if thread.thread_id == this_thread_id}}
(<i>this</i> request)
{{endif}}</td>
<td>
<b>{{thread.thread_id}}
{{if allow_kill}}
<form action="{{script_name}}/kill" method="POST"
style="display: inline">
<input type="hidden" name="thread_id" value="{{thread.thread_id}}">
<input type="submit" value="kill">
</form>
{{endif}}
</b>
</td>
</tr>
<tr>
<td>Time processing request</td>
<td>{{thread.time_html|html}}</td>
</tr>
<tr>
<td>URI</td>
<td>{{if thread.uri == 'unknown'}}
unknown
{{else}}<a href="{{thread.uri}}">{{thread.uri_short}}</a>
{{endif}}
</td>
<tr>
<td colspan="2" class="bottom">
<a href="#" class="button" style="width: 9em; display: block"
onclick="
var el = document.getElementById('environ-{{thread.thread_id}}');
if (el.style.display) {
el.style.display = '';
this.innerHTML = \'&#9662; Hide environ\';
} else {
el.style.display = 'none';
this.innerHTML = \'&#9656; Show environ\';
}
return false
">&#9656; Show environ</a>
<div id="environ-{{thread.thread_id}}" style="display: none">
{{if thread.environ:}}
<table class="environ">
{{for loop, item in looper(sorted(thread.environ.items()))}}
{{py:key, value=item}}
<tr>
<td {{if loop.last}}class="bottom"{{endif}}>{{key}}</td>
<td {{if loop.last}}class="bottom"{{endif}}>{{value}}</td>
</tr>
{{endfor}}
</table>
{{else}}
Thread is in process of starting
{{endif}}
</div>
{{if thread.traceback}}
<a href="#" class="button" style="width: 9em; display: block"
onclick="
var el = document.getElementById('traceback-{{thread.thread_id}}');
if (el.style.display) {
el.style.display = '';
this.innerHTML = \'&#9662; Hide traceback\';
} else {
el.style.display = 'none';
this.innerHTML = \'&#9656; Show traceback\';
}
return false
">&#9656; Show traceback</a>
<div id="traceback-{{thread.thread_id}}" style="display: none">
<pre class="traceback">{{thread.traceback}}</pre>
</div>
{{endif}}
</td>
</tr>
</table>
{{endfor}}
</body>
</html>
''', name='watchthreads.page_template')
class WatchThreads(object):
"""
Application that watches the threads in ``paste.httpserver``,
showing the length each thread has been working on a request.
If allow_kill is true, then you can kill errant threads through
this application.
This application can expose private information (specifically in
the environment, like cookies), so it should be protected.
"""
def __init__(self, allow_kill=False):
self.allow_kill = allow_kill
def __call__(self, environ, start_response):
if 'paste.httpserver.thread_pool' not in environ:
start_response('403 Forbidden', [('Content-type', 'text/plain')])
return ['You must use the threaded Paste HTTP server to use this application']
if environ.get('PATH_INFO') == '/kill':
return self.kill(environ, start_response)
else:
return self.show(environ, start_response)
def show(self, environ, start_response):
start_response('200 OK', [('Content-type', 'text/html')])
form = parse_formvars(environ)
if form.get('kill'):
kill_thread_id = form['kill']
else:
kill_thread_id = None
thread_pool = environ['paste.httpserver.thread_pool']
nworkers = thread_pool.nworkers
now = time.time()
workers = thread_pool.worker_tracker.items()
workers.sort(key=lambda v: v[1][0])
threads = []
for thread_id, (time_started, worker_environ) in workers:
thread = bunch()
threads.append(thread)
if worker_environ:
thread.uri = construct_url(worker_environ)
else:
thread.uri = 'unknown'
thread.thread_id = thread_id
thread.time_html = format_time(now-time_started)
thread.uri_short = shorten(thread.uri)
thread.environ = worker_environ
thread.traceback = traceback_thread(thread_id)
page = page_template.substitute(
title="Thread Pool Worker Tracker",
nworkers=nworkers,
actual_workers=len(thread_pool.workers),
nworkers_used=len(workers),
script_name=environ['SCRIPT_NAME'],
kill_thread_id=kill_thread_id,
allow_kill=self.allow_kill,
threads=threads,
this_thread_id=get_ident(),
track_threads=thread_pool.track_threads())
return [page]
def kill(self, environ, start_response):
if not self.allow_kill:
exc = httpexceptions.HTTPForbidden(
'Killing threads has not been enabled. Shame on you '
'for trying!')
return exc(environ, start_response)
vars = parse_formvars(environ)
thread_id = int(vars['thread_id'])
thread_pool = environ['paste.httpserver.thread_pool']
if thread_id not in thread_pool.worker_tracker:
exc = httpexceptions.PreconditionFailed(
'You tried to kill thread %s, but it is not working on '
'any requests' % thread_id)
return exc(environ, start_response)
thread_pool.kill_worker(thread_id)
script_name = environ['SCRIPT_NAME'] or '/'
exc = httpexceptions.HTTPFound(
headers=[('Location', script_name+'?kill=%s' % thread_id)])
return exc(environ, start_response)
def traceback_thread(thread_id):
"""
Returns a plain-text traceback of the given thread, or None if it
can't get a traceback.
"""
if not hasattr(sys, '_current_frames'):
# Only 2.5 has support for this, with this special function
return None
frames = sys._current_frames()
if not thread_id in frames:
return None
frame = frames[thread_id]
out = StringIO()
traceback.print_stack(frame, file=out)
return out.getvalue()
hide_keys = ['paste.httpserver.thread_pool']
def format_environ(environ):
if environ is None:
return environ_template.substitute(
key='---',
value='No environment registered for this thread yet')
environ_rows = []
for key, value in sorted(environ.items()):
if key in hide_keys:
continue
try:
if key.upper() != key:
value = repr(value)
environ_rows.append(
environ_template.substitute(
key=cgi.escape(str(key)),
value=cgi.escape(str(value))))
except Exception, e:
environ_rows.append(
environ_template.substitute(
key=cgi.escape(str(key)),
value='Error in <code>repr()</code>: %s' % e))
return ''.join(environ_rows)
def format_time(time_length):
if time_length >= 60*60:
# More than an hour
time_string = '%i:%02i:%02i' % (int(time_length/60/60),
int(time_length/60) % 60,
time_length % 60)
elif time_length >= 120:
time_string = '%i:%02i' % (int(time_length/60),
time_length % 60)
elif time_length > 60:
time_string = '%i sec' % time_length
elif time_length > 1:
time_string = '%0.1f sec' % time_length
else:
time_string = '%0.2f sec' % time_length
if time_length < 5:
return time_string
elif time_length < 120:
return '<span style="color: #900">%s</span>' % time_string
else:
return '<span style="background-color: #600; color: #fff">%s</span>' % time_string
def shorten(s):
if len(s) > 60:
return s[:40]+'...'+s[-10:]
else:
return s
def make_watch_threads(global_conf, allow_kill=False):
from paste.deploy.converters import asbool
return WatchThreads(allow_kill=asbool(allow_kill))
make_watch_threads.__doc__ = WatchThreads.__doc__
def make_bad_app(global_conf, pause=0):
pause = int(pause)
def bad_app(environ, start_response):
import thread
if pause:
time.sleep(pause)
else:
count = 0
while 1:
print "I'm alive %s (%s)" % (count, thread.get_ident())
time.sleep(10)
count += 1
start_response('200 OK', [('content-type', 'text/plain')])
return ['OK, paused %s seconds' % pause]
return bad_app

View File

@@ -0,0 +1,121 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
"""
Middleware that tests the validity of all generated HTML using the
`WDG HTML Validator <http://www.htmlhelp.com/tools/validator/>`_
"""
from cStringIO import StringIO
try:
import subprocess
except ImportError:
from paste.util import subprocess24 as subprocess
from paste.response import header_value
import re
import cgi
__all__ = ['WDGValidateMiddleware']
class WDGValidateMiddleware(object):
"""
Middleware that checks HTML and appends messages about the validity of
the HTML. Uses: http://www.htmlhelp.com/tools/validator/ -- interacts
with the command line client. Use the configuration ``wdg_path`` to
override the path (default: looks for ``validate`` in $PATH).
To install, in your web context's __init__.py::
def urlparser_wrap(environ, start_response, app):
return wdg_validate.WDGValidateMiddleware(app)(
environ, start_response)
Or in your configuration::
middleware.append('paste.wdg_validate.WDGValidateMiddleware')
"""
_end_body_regex = re.compile(r'</body>', re.I)
def __init__(self, app, global_conf=None, wdg_path='validate'):
self.app = app
self.wdg_path = wdg_path
def __call__(self, environ, start_response):
output = StringIO()
response = []
def writer_start_response(status, headers, exc_info=None):
response.extend((status, headers))
start_response(status, headers, exc_info)
return output.write
app_iter = self.app(environ, writer_start_response)
try:
for s in app_iter:
output.write(s)
finally:
if hasattr(app_iter, 'close'):
app_iter.close()
page = output.getvalue()
status, headers = response
v = header_value(headers, 'content-type') or ''
if (not v.startswith('text/html')
and not v.startswith('text/xhtml')
and not v.startswith('application/xhtml')):
# Can't validate
# @@: Should validate CSS too... but using what?
return [page]
ops = []
if v.startswith('text/xhtml+xml'):
ops.append('--xml')
# @@: Should capture encoding too
html_errors = self.call_wdg_validate(
self.wdg_path, ops, page)
if html_errors:
page = self.add_error(page, html_errors)[0]
headers.remove(
('Content-Length',
str(header_value(headers, 'content-length'))))
headers.append(('Content-Length', str(len(page))))
return [page]
def call_wdg_validate(self, wdg_path, ops, page):
if subprocess is None:
raise ValueError(
"This middleware requires the subprocess module from "
"Python 2.4")
proc = subprocess.Popen([wdg_path] + ops,
shell=False,
close_fds=True,
stdout=subprocess.PIPE,
stdin=subprocess.PIPE,
stderr=subprocess.STDOUT)
stdout = proc.communicate(page)[0]
proc.wait()
return stdout
def add_error(self, html_page, html_errors):
add_text = ('<pre style="background-color: #ffd; color: #600; '
'border: 1px solid #000;">%s</pre>'
% cgi.escape(html_errors))
match = self._end_body_regex.search(html_page)
if match:
return [html_page[:match.start()]
+ add_text
+ html_page[match.start():]]
else:
return [html_page + add_text]
def make_wdg_validate_middleware(
app, global_conf, wdg_path='validate'):
"""
Wraps the application in the WDG validator from
http://www.htmlhelp.com/tools/validator/
Validation errors are appended to the text of each page.
You can configure this by giving the path to the validate
executable (by default picked up from $PATH)
"""
return WDGValidateMiddleware(
app, global_conf, wdg_path=wdg_path)

View File

@@ -0,0 +1,383 @@
# (c) 2005-2006 James Gardner <james@pythonweb.org>
# This module is part of the Python Paste Project and is released under
# the MIT License: http://www.opensource.org/licenses/mit-license.php
"""
Middleware to display error documents for certain status codes
The middleware in this module can be used to intercept responses with
specified status codes and internally forward the request to an appropriate
URL where the content can be displayed to the user as an error document.
"""
import warnings
import sys
from urlparse import urlparse
from paste.recursive import ForwardRequestException, RecursiveMiddleware, RecursionLoop
from paste.util import converters
from paste.response import replace_header
def forward(app, codes):
"""
Intercepts a response with a particular status code and returns the
content from a specified URL instead.
The arguments are:
``app``
The WSGI application or middleware chain.
``codes``
A dictionary of integer status codes and the URL to be displayed
if the response uses that code.
For example, you might want to create a static file to display a
"File Not Found" message at the URL ``/error404.html`` and then use
``forward`` middleware to catch all 404 status codes and display the page
you created. In this example ``app`` is your exisiting WSGI
applicaiton::
from paste.errordocument import forward
app = forward(app, codes={404:'/error404.html'})
"""
for code in codes:
if not isinstance(code, int):
raise TypeError('All status codes should be type int. '
'%s is not valid'%repr(code))
def error_codes_mapper(code, message, environ, global_conf, codes):
if codes.has_key(code):
return codes[code]
else:
return None
#return _StatusBasedRedirect(app, error_codes_mapper, codes=codes)
return RecursiveMiddleware(
StatusBasedForward(
app,
error_codes_mapper,
codes=codes,
)
)
class StatusKeeper(object):
def __init__(self, app, status, url, headers):
self.app = app
self.status = status
self.url = url
self.headers = headers
def __call__(self, environ, start_response):
def keep_status_start_response(status, headers, exc_info=None):
for header, value in headers:
if header.lower() == 'set-cookie':
self.headers.append((header, value))
else:
replace_header(self.headers, header, value)
return start_response(self.status, self.headers, exc_info)
parts = self.url.split('?')
environ['PATH_INFO'] = parts[0]
if len(parts) > 1:
environ['QUERY_STRING'] = parts[1]
else:
environ['QUERY_STRING'] = ''
#raise Exception(self.url, self.status)
try:
return self.app(environ, keep_status_start_response)
except RecursionLoop, e:
environ['wsgi.errors'].write('Recursion error getting error page: %s\n' % e)
keep_status_start_response('500 Server Error', [('Content-type', 'text/plain')], sys.exc_info())
return ['Error: %s. (Error page could not be fetched)'
% self.status]
class StatusBasedForward(object):
"""
Middleware that lets you test a response against a custom mapper object to
programatically determine whether to internally forward to another URL and
if so, which URL to forward to.
If you don't need the full power of this middleware you might choose to use
the simpler ``forward`` middleware instead.
The arguments are:
``app``
The WSGI application or middleware chain.
``mapper``
A callable that takes a status code as the
first parameter, a message as the second, and accepts optional environ,
global_conf and named argments afterwards. It should return a
URL to forward to or ``None`` if the code is not to be intercepted.
``global_conf``
Optional default configuration from your config file. If ``debug`` is
set to ``true`` a message will be written to ``wsgi.errors`` on each
internal forward stating the URL forwarded to.
``**params``
Optional, any other configuration and extra arguments you wish to
pass which will in turn be passed back to the custom mapper object.
Here is an example where a ``404 File Not Found`` status response would be
redirected to the URL ``/error?code=404&message=File%20Not%20Found``. This
could be useful for passing the status code and message into another
application to display an error document:
.. code-block:: python
from paste.errordocument import StatusBasedForward
from paste.recursive import RecursiveMiddleware
from urllib import urlencode
def error_mapper(code, message, environ, global_conf, kw)
if code in [404, 500]:
params = urlencode({'message':message, 'code':code})
url = '/error?'%(params)
return url
else:
return None
app = RecursiveMiddleware(
StatusBasedForward(app, mapper=error_mapper),
)
"""
def __init__(self, app, mapper, global_conf=None, **params):
if global_conf is None:
global_conf = {}
# @@: global_conf shouldn't really come in here, only in a
# separate make_status_based_forward function
if global_conf:
self.debug = converters.asbool(global_conf.get('debug', False))
else:
self.debug = False
self.application = app
self.mapper = mapper
self.global_conf = global_conf
self.params = params
def __call__(self, environ, start_response):
url = []
writer = []
def change_response(status, headers, exc_info=None):
status_code = status.split(' ')
try:
code = int(status_code[0])
except (ValueError, TypeError):
raise Exception(
'StatusBasedForward middleware '
'received an invalid status code %s'%repr(status_code[0])
)
message = ' '.join(status_code[1:])
new_url = self.mapper(
code,
message,
environ,
self.global_conf,
**self.params
)
if not (new_url == None or isinstance(new_url, str)):
raise TypeError(
'Expected the url to internally '
'redirect to in the StatusBasedForward mapper'
'to be a string or None, not %r' % new_url)
if new_url:
url.append([new_url, status, headers])
# We have to allow the app to write stuff, even though
# we'll ignore it:
return [].append
else:
return start_response(status, headers, exc_info)
app_iter = self.application(environ, change_response)
if url:
if hasattr(app_iter, 'close'):
app_iter.close()
def factory(app):
return StatusKeeper(app, status=url[0][1], url=url[0][0],
headers=url[0][2])
raise ForwardRequestException(factory=factory)
else:
return app_iter
def make_errordocument(app, global_conf, **kw):
"""
Paste Deploy entry point to create a error document wrapper.
Use like::
[filter-app:main]
use = egg:Paste#errordocument
next = real-app
500 = /lib/msg/500.html
404 = /lib/msg/404.html
"""
map = {}
for status, redir_loc in kw.items():
try:
status = int(status)
except ValueError:
raise ValueError('Bad status code: %r' % status)
map[status] = redir_loc
forwarder = forward(app, map)
return forwarder
__pudge_all__ = [
'forward',
'make_errordocument',
'empty_error',
'make_empty_error',
'StatusBasedForward',
]
###############################################################################
## Deprecated
###############################################################################
def custom_forward(app, mapper, global_conf=None, **kw):
"""
Deprectated; use StatusBasedForward instead.
"""
warnings.warn(
"errordocuments.custom_forward has been deprecated; please "
"use errordocuments.StatusBasedForward",
DeprecationWarning, 2)
if global_conf is None:
global_conf = {}
return _StatusBasedRedirect(app, mapper, global_conf, **kw)
class _StatusBasedRedirect(object):
"""
Deprectated; use StatusBasedForward instead.
"""
def __init__(self, app, mapper, global_conf=None, **kw):
warnings.warn(
"errordocuments._StatusBasedRedirect has been deprecated; please "
"use errordocuments.StatusBasedForward",
DeprecationWarning, 2)
if global_conf is None:
global_conf = {}
self.application = app
self.mapper = mapper
self.global_conf = global_conf
self.kw = kw
self.fallback_template = """
<html>
<head>
<title>Error %(code)s</title>
</html>
<body>
<h1>Error %(code)s</h1>
<p>%(message)s</p>
<hr>
<p>
Additionally an error occurred trying to produce an
error document. A description of the error was logged
to <tt>wsgi.errors</tt>.
</p>
</body>
</html>
"""
def __call__(self, environ, start_response):
url = []
code_message = []
try:
def change_response(status, headers, exc_info=None):
new_url = None
parts = status.split(' ')
try:
code = int(parts[0])
except (ValueError, TypeError):
raise Exception(
'_StatusBasedRedirect middleware '
'received an invalid status code %s'%repr(parts[0])
)
message = ' '.join(parts[1:])
new_url = self.mapper(
code,
message,
environ,
self.global_conf,
self.kw
)
if not (new_url == None or isinstance(new_url, str)):
raise TypeError(
'Expected the url to internally '
'redirect to in the _StatusBasedRedirect error_mapper'
'to be a string or None, not %s'%repr(new_url)
)
if new_url:
url.append(new_url)
code_message.append([code, message])
return start_response(status, headers, exc_info)
app_iter = self.application(environ, change_response)
except:
try:
import sys
error = str(sys.exc_info()[1])
except:
error = ''
try:
code, message = code_message[0]
except:
code, message = ['', '']
environ['wsgi.errors'].write(
'Error occurred in _StatusBasedRedirect '
'intercepting the response: '+str(error)
)
return [self.fallback_template
% {'message': message, 'code': code}]
else:
if url:
url_ = url[0]
new_environ = {}
for k, v in environ.items():
if k != 'QUERY_STRING':
new_environ['QUERY_STRING'] = urlparse(url_)[4]
else:
new_environ[k] = v
class InvalidForward(Exception):
pass
def eat_start_response(status, headers, exc_info=None):
"""
We don't want start_response to do anything since it
has already been called
"""
if status[:3] != '200':
raise InvalidForward(
"The URL %s to internally forward "
"to in order to create an error document did not "
"return a '200' status code." % url_
)
forward = environ['paste.recursive.forward']
old_start_response = forward.start_response
forward.start_response = eat_start_response
try:
app_iter = forward(url_, new_environ)
except InvalidForward, e:
code, message = code_message[0]
environ['wsgi.errors'].write(
'Error occurred in '
'_StatusBasedRedirect redirecting '
'to new URL: '+str(url[0])
)
return [
self.fallback_template%{
'message':message,
'code':code,
}
]
else:
forward.start_response = old_start_response
return app_iter
else:
return app_iter

View File

@@ -0,0 +1,7 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
"""
An exception handler for interactive debugging
"""
from paste.evalexception.middleware import EvalException

View File

@@ -0,0 +1,68 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
from cStringIO import StringIO
import traceback
import threading
import pdb
import sys
exec_lock = threading.Lock()
class EvalContext(object):
"""
Class that represents a interactive interface. It has its own
namespace. Use eval_context.exec_expr(expr) to run commands; the
output of those commands is returned, as are print statements.
This is essentially what doctest does, and is taken directly from
doctest.
"""
def __init__(self, namespace, globs):
self.namespace = namespace
self.globs = globs
def exec_expr(self, s):
out = StringIO()
exec_lock.acquire()
save_stdout = sys.stdout
try:
debugger = _OutputRedirectingPdb(save_stdout)
debugger.reset()
pdb.set_trace = debugger.set_trace
sys.stdout = out
try:
code = compile(s, '<web>', "single", 0, 1)
exec code in self.namespace, self.globs
debugger.set_continue()
except KeyboardInterrupt:
raise
except:
traceback.print_exc(file=out)
debugger.set_continue()
finally:
sys.stdout = save_stdout
exec_lock.release()
return out.getvalue()
# From doctest
class _OutputRedirectingPdb(pdb.Pdb):
"""
A specialized version of the python debugger that redirects stdout
to a given stream when interacting with the user. Stdout is *not*
redirected when traced code is executed.
"""
def __init__(self, out):
self.__out = out
pdb.Pdb.__init__(self)
def trace_dispatch(self, *args):
# Redirect stdout to the given stream.
save_stdout = sys.stdout
sys.stdout = self.__out
# Call Pdb's trace dispatch method.
try:
return pdb.Pdb.trace_dispatch(self, *args)
finally:
sys.stdout = save_stdout

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,161 @@
function showFrame(anchor) {
var tbid = anchor.getAttribute('tbid');
var expanded = anchor.expanded;
if (expanded) {
MochiKit.DOM.hideElement(anchor.expandedElement);
anchor.expanded = false;
_swapImage(anchor);
return false;
}
anchor.expanded = true;
if (anchor.expandedElement) {
MochiKit.DOM.showElement(anchor.expandedElement);
_swapImage(anchor);
$('debug_input_'+tbid).focus();
return false;
}
var url = debug_base
+ '/show_frame?tbid=' + tbid
+ '&debugcount=' + debug_count;
var d = MochiKit.Async.doSimpleXMLHttpRequest(url);
d.addCallbacks(function (data) {
var el = MochiKit.DOM.DIV({});
anchor.parentNode.insertBefore(el, anchor.nextSibling);
el.innerHTML = data.responseText;
anchor.expandedElement = el;
_swapImage(anchor);
$('debug_input_'+tbid).focus();
}, function (error) {
showError(error.req.responseText);
});
return false;
}
function _swapImage(anchor) {
var el = anchor.getElementsByTagName('IMG')[0];
if (anchor.expanded) {
var img = 'minus.jpg';
} else {
var img = 'plus.jpg';
}
el.src = debug_base + '/media/' + img;
}
function submitInput(button, tbid) {
var input = $(button.getAttribute('input-from'));
var output = $(button.getAttribute('output-to'));
var url = debug_base
+ '/exec_input';
var history = input.form.history;
input.historyPosition = 0;
if (! history) {
history = input.form.history = [];
}
history.push(input.value);
var vars = {
tbid: tbid,
debugcount: debug_count,
input: input.value
};
MochiKit.DOM.showElement(output);
var d = MochiKit.Async.doSimpleXMLHttpRequest(url, vars);
d.addCallbacks(function (data) {
var result = data.responseText;
output.innerHTML += result;
input.value = '';
input.focus();
}, function (error) {
showError(error.req.responseText);
});
return false;
}
function showError(msg) {
var el = $('error-container');
if (el.innerHTML) {
el.innerHTML += '<hr noshade>\n' + msg;
} else {
el.innerHTML = msg;
}
MochiKit.DOM.showElement('error-area');
}
function clearError() {
var el = $('error-container');
el.innerHTML = '';
MochiKit.DOM.hideElement('error-area');
}
function expandInput(button) {
var input = button.form.elements.input;
stdops = {
name: 'input',
style: 'width: 100%',
autocomplete: 'off'
};
if (input.tagName == 'INPUT') {
var newEl = MochiKit.DOM.TEXTAREA(stdops);
var text = 'Contract';
} else {
stdops['type'] = 'text';
stdops['onkeypress'] = 'upArrow(this)';
var newEl = MochiKit.DOM.INPUT(stdops);
var text = 'Expand';
}
newEl.value = input.value;
newEl.id = input.id;
MochiKit.DOM.swapDOM(input, newEl);
newEl.focus();
button.value = text;
return false;
}
function upArrow(input, event) {
if (window.event) {
event = window.event;
}
if (event.keyCode != 38 && event.keyCode != 40) {
// not an up- or down-arrow
return true;
}
var dir = event.keyCode == 38 ? 1 : -1;
var history = input.form.history;
if (! history) {
history = input.form.history = [];
}
var pos = input.historyPosition || 0;
if (! pos && dir == -1) {
return true;
}
if (! pos && input.value) {
history.push(input.value);
pos = 1;
}
pos += dir;
if (history.length-pos < 0) {
pos = 1;
}
if (history.length-pos > history.length-1) {
input.value = '';
return true;
}
input.historyPosition = pos;
var line = history[history.length-pos];
input.value = line;
}
function expandLong(anchor) {
var span = anchor;
while (span) {
if (span.style && span.style.display == 'none') {
break;
}
span = span.nextSibling;
}
if (! span) {
return false;
}
MochiKit.DOM.showElement(span);
MochiKit.DOM.hideElement(anchor);
return false;
}

Binary file not shown.

After

Width:  |  Height:  |  Size: 359 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 361 B

View File

@@ -0,0 +1,610 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
"""
Exception-catching middleware that allows interactive debugging.
This middleware catches all unexpected exceptions. A normal
traceback, like produced by
``paste.exceptions.errormiddleware.ErrorMiddleware`` is given, plus
controls to see local variables and evaluate expressions in a local
context.
This can only be used in single-process environments, because
subsequent requests must go back to the same process that the
exception originally occurred in. Threaded or non-concurrent
environments both work.
This shouldn't be used in production in any way. That would just be
silly.
If calling from an XMLHttpRequest call, if the GET variable ``_`` is
given then it will make the response more compact (and less
Javascripty), since if you use innerHTML it'll kill your browser. You
can look for the header X-Debug-URL in your 500 responses if you want
to see the full debuggable traceback. Also, this URL is printed to
``wsgi.errors``, so you can open it up in another browser window.
"""
import sys
import os
import cgi
import traceback
from cStringIO import StringIO
import pprint
import itertools
import time
import re
from paste.exceptions import errormiddleware, formatter, collector
from paste import wsgilib
from paste import urlparser
from paste import httpexceptions
from paste import registry
from paste import request
from paste import response
import evalcontext
limit = 200
def html_quote(v):
"""
Escape HTML characters, plus translate None to ''
"""
if v is None:
return ''
return cgi.escape(str(v), 1)
def preserve_whitespace(v, quote=True):
"""
Quote a value for HTML, preserving whitespace (translating
newlines to ``<br>`` and multiple spaces to use ``&nbsp;``).
If ``quote`` is true, then the value will be HTML quoted first.
"""
if quote:
v = html_quote(v)
v = v.replace('\n', '<br>\n')
v = re.sub(r'()( +)', _repl_nbsp, v)
v = re.sub(r'(\n)( +)', _repl_nbsp, v)
v = re.sub(r'^()( +)', _repl_nbsp, v)
return '<code>%s</code>' % v
def _repl_nbsp(match):
if len(match.group(2)) == 1:
return '&nbsp;'
return match.group(1) + '&nbsp;' * (len(match.group(2))-1) + ' '
def simplecatcher(application):
"""
A simple middleware that catches errors and turns them into simple
tracebacks.
"""
def simplecatcher_app(environ, start_response):
try:
return application(environ, start_response)
except:
out = StringIO()
traceback.print_exc(file=out)
start_response('500 Server Error',
[('content-type', 'text/html')],
sys.exc_info())
res = out.getvalue()
return ['<h3>Error</h3><pre>%s</pre>'
% html_quote(res)]
return simplecatcher_app
def wsgiapp():
"""
Turns a function or method into a WSGI application.
"""
def decorator(func):
def wsgiapp_wrapper(*args):
# we get 3 args when this is a method, two when it is
# a function :(
if len(args) == 3:
environ = args[1]
start_response = args[2]
args = [args[0]]
else:
environ, start_response = args
args = []
def application(environ, start_response):
form = wsgilib.parse_formvars(environ,
include_get_vars=True)
headers = response.HeaderDict(
{'content-type': 'text/html',
'status': '200 OK'})
form['environ'] = environ
form['headers'] = headers
res = func(*args, **form.mixed())
status = headers.pop('status')
start_response(status, headers.headeritems())
return [res]
app = httpexceptions.make_middleware(application)
app = simplecatcher(app)
return app(environ, start_response)
wsgiapp_wrapper.exposed = True
return wsgiapp_wrapper
return decorator
def get_debug_info(func):
"""
A decorator (meant to be used under ``wsgiapp()``) that resolves
the ``debugcount`` variable to a ``DebugInfo`` object (or gives an
error if it can't be found).
"""
def debug_info_replacement(self, **form):
try:
if 'debugcount' not in form:
raise ValueError('You must provide a debugcount parameter')
debugcount = form.pop('debugcount')
try:
debugcount = int(debugcount)
except ValueError:
raise ValueError('Bad value for debugcount')
if debugcount not in self.debug_infos:
raise ValueError(
'Debug %s no longer found (maybe it has expired?)'
% debugcount)
debug_info = self.debug_infos[debugcount]
return func(self, debug_info=debug_info, **form)
except ValueError, e:
form['headers']['status'] = '500 Server Error'
return '<html>There was an error: %s</html>' % html_quote(e)
return debug_info_replacement
debug_counter = itertools.count(int(time.time()))
def get_debug_count(environ):
"""
Return the unique debug count for the current request
"""
if 'paste.evalexception.debug_count' in environ:
return environ['paste.evalexception.debug_count']
else:
environ['paste.evalexception.debug_count'] = next = debug_counter.next()
return next
class EvalException(object):
def __init__(self, application, global_conf=None,
xmlhttp_key=None):
self.application = application
self.debug_infos = {}
if xmlhttp_key is None:
if global_conf is None:
xmlhttp_key = '_'
else:
xmlhttp_key = global_conf.get('xmlhttp_key', '_')
self.xmlhttp_key = xmlhttp_key
def __call__(self, environ, start_response):
assert not environ['wsgi.multiprocess'], (
"The EvalException middleware is not usable in a "
"multi-process environment")
environ['paste.evalexception'] = self
if environ.get('PATH_INFO', '').startswith('/_debug/'):
return self.debug(environ, start_response)
else:
return self.respond(environ, start_response)
def debug(self, environ, start_response):
assert request.path_info_pop(environ) == '_debug'
next_part = request.path_info_pop(environ)
method = getattr(self, next_part, None)
if not method:
exc = httpexceptions.HTTPNotFound(
'%r not found when parsing %r'
% (next_part, wsgilib.construct_url(environ)))
return exc.wsgi_application(environ, start_response)
if not getattr(method, 'exposed', False):
exc = httpexceptions.HTTPForbidden(
'%r not allowed' % next_part)
return exc.wsgi_application(environ, start_response)
return method(environ, start_response)
def media(self, environ, start_response):
"""
Static path where images and other files live
"""
app = urlparser.StaticURLParser(
os.path.join(os.path.dirname(__file__), 'media'))
return app(environ, start_response)
media.exposed = True
def mochikit(self, environ, start_response):
"""
Static path where MochiKit lives
"""
app = urlparser.StaticURLParser(
os.path.join(os.path.dirname(__file__), 'mochikit'))
return app(environ, start_response)
mochikit.exposed = True
def summary(self, environ, start_response):
"""
Returns a JSON-format summary of all the cached
exception reports
"""
start_response('200 OK', [('Content-type', 'text/x-json')])
data = [];
items = self.debug_infos.values()
items.sort(lambda a, b: cmp(a.created, b.created))
data = [item.json() for item in items]
return [repr(data)]
summary.exposed = True
def view(self, environ, start_response):
"""
View old exception reports
"""
id = int(request.path_info_pop(environ))
if id not in self.debug_infos:
start_response(
'500 Server Error',
[('Content-type', 'text/html')])
return [
"Traceback by id %s does not exist (maybe "
"the server has been restarted?)"
% id]
debug_info = self.debug_infos[id]
return debug_info.wsgi_application(environ, start_response)
view.exposed = True
def make_view_url(self, environ, base_path, count):
return base_path + '/_debug/view/%s' % count
#@wsgiapp()
#@get_debug_info
def show_frame(self, tbid, debug_info, **kw):
frame = debug_info.frame(int(tbid))
vars = frame.tb_frame.f_locals
if vars:
registry.restorer.restoration_begin(debug_info.counter)
local_vars = make_table(vars)
registry.restorer.restoration_end()
else:
local_vars = 'No local vars'
return input_form(tbid, debug_info) + local_vars
show_frame = wsgiapp()(get_debug_info(show_frame))
#@wsgiapp()
#@get_debug_info
def exec_input(self, tbid, debug_info, input, **kw):
if not input.strip():
return ''
input = input.rstrip() + '\n'
frame = debug_info.frame(int(tbid))
vars = frame.tb_frame.f_locals
glob_vars = frame.tb_frame.f_globals
context = evalcontext.EvalContext(vars, glob_vars)
registry.restorer.restoration_begin(debug_info.counter)
output = context.exec_expr(input)
registry.restorer.restoration_end()
input_html = formatter.str2html(input)
return ('<code style="color: #060">&gt;&gt;&gt;</code> '
'<code>%s</code><br>\n%s'
% (preserve_whitespace(input_html, quote=False),
preserve_whitespace(output)))
exec_input = wsgiapp()(get_debug_info(exec_input))
def respond(self, environ, start_response):
if environ.get('paste.throw_errors'):
return self.application(environ, start_response)
base_path = request.construct_url(environ, with_path_info=False,
with_query_string=False)
environ['paste.throw_errors'] = True
started = []
def detect_start_response(status, headers, exc_info=None):
try:
return start_response(status, headers, exc_info)
except:
raise
else:
started.append(True)
try:
__traceback_supplement__ = errormiddleware.Supplement, self, environ
app_iter = self.application(environ, detect_start_response)
try:
return_iter = list(app_iter)
return return_iter
finally:
if hasattr(app_iter, 'close'):
app_iter.close()
except:
exc_info = sys.exc_info()
for expected in environ.get('paste.expected_exceptions', []):
if isinstance(exc_info[1], expected):
raise
# Tell the Registry to save its StackedObjectProxies current state
# for later restoration
registry.restorer.save_registry_state(environ)
count = get_debug_count(environ)
view_uri = self.make_view_url(environ, base_path, count)
if not started:
headers = [('content-type', 'text/html')]
headers.append(('X-Debug-URL', view_uri))
start_response('500 Internal Server Error',
headers,
exc_info)
environ['wsgi.errors'].write('Debug at: %s\n' % view_uri)
exc_data = collector.collect_exception(*exc_info)
debug_info = DebugInfo(count, exc_info, exc_data, base_path,
environ, view_uri)
assert count not in self.debug_infos
self.debug_infos[count] = debug_info
if self.xmlhttp_key:
get_vars = wsgilib.parse_querystring(environ)
if dict(get_vars).get(self.xmlhttp_key):
exc_data = collector.collect_exception(*exc_info)
html = formatter.format_html(
exc_data, include_hidden_frames=False,
include_reusable=False, show_extra_data=False)
return [html]
# @@: it would be nice to deal with bad content types here
return debug_info.content()
def exception_handler(self, exc_info, environ):
simple_html_error = False
if self.xmlhttp_key:
get_vars = wsgilib.parse_querystring(environ)
if dict(get_vars).get(self.xmlhttp_key):
simple_html_error = True
return errormiddleware.handle_exception(
exc_info, environ['wsgi.errors'],
html=True,
debug_mode=True,
simple_html_error=simple_html_error)
class DebugInfo(object):
def __init__(self, counter, exc_info, exc_data, base_path,
environ, view_uri):
self.counter = counter
self.exc_data = exc_data
self.base_path = base_path
self.environ = environ
self.view_uri = view_uri
self.created = time.time()
self.exc_type, self.exc_value, self.tb = exc_info
__exception_formatter__ = 1
self.frames = []
n = 0
tb = self.tb
while tb is not None and (limit is None or n < limit):
if tb.tb_frame.f_locals.get('__exception_formatter__'):
# Stop recursion. @@: should make a fake ExceptionFrame
break
self.frames.append(tb)
tb = tb.tb_next
n += 1
def json(self):
"""Return the JSON-able representation of this object"""
return {
'uri': self.view_uri,
'created': time.strftime('%c', time.gmtime(self.created)),
'created_timestamp': self.created,
'exception_type': str(self.exc_type),
'exception': str(self.exc_value),
}
def frame(self, tbid):
for frame in self.frames:
if id(frame) == tbid:
return frame
else:
raise ValueError, (
"No frame by id %s found from %r" % (tbid, self.frames))
def wsgi_application(self, environ, start_response):
start_response('200 OK', [('content-type', 'text/html')])
return self.content()
def content(self):
html = format_eval_html(self.exc_data, self.base_path, self.counter)
head_html = (formatter.error_css + formatter.hide_display_js)
head_html += self.eval_javascript()
repost_button = make_repost_button(self.environ)
page = error_template % {
'repost_button': repost_button or '',
'head_html': head_html,
'body': html}
return [page]
def eval_javascript(self):
base_path = self.base_path + '/_debug'
return (
'<script type="text/javascript" src="%s/media/MochiKit.packed.js">'
'</script>\n'
'<script type="text/javascript" src="%s/media/debug.js">'
'</script>\n'
'<script type="text/javascript">\n'
'debug_base = %r;\n'
'debug_count = %r;\n'
'</script>\n'
% (base_path, base_path, base_path, self.counter))
class EvalHTMLFormatter(formatter.HTMLFormatter):
def __init__(self, base_path, counter, **kw):
super(EvalHTMLFormatter, self).__init__(**kw)
self.base_path = base_path
self.counter = counter
def format_source_line(self, filename, frame):
line = formatter.HTMLFormatter.format_source_line(
self, filename, frame)
return (line +
' <a href="#" class="switch_source" '
'tbid="%s" onClick="return showFrame(this)">&nbsp; &nbsp; '
'<img src="%s/_debug/media/plus.jpg" border=0 width=9 '
'height=9> &nbsp; &nbsp;</a>'
% (frame.tbid, self.base_path))
def make_table(items):
if isinstance(items, dict):
items = items.items()
items.sort()
rows = []
i = 0
for name, value in items:
i += 1
out = StringIO()
try:
pprint.pprint(value, out)
except Exception, e:
print >> out, 'Error: %s' % e
value = html_quote(out.getvalue())
if len(value) > 100:
# @@: This can actually break the HTML :(
# should I truncate before quoting?
orig_value = value
value = value[:100]
value += '<a class="switch_source" style="background-color: #999" href="#" onclick="return expandLong(this)">...</a>'
value += '<span style="display: none">%s</span>' % orig_value[100:]
value = formatter.make_wrappable(value)
if i % 2:
attr = ' class="even"'
else:
attr = ' class="odd"'
rows.append('<tr%s style="vertical-align: top;"><td>'
'<b>%s</b></td><td style="overflow: auto">%s<td></tr>'
% (attr, html_quote(name),
preserve_whitespace(value, quote=False)))
return '<table>%s</table>' % (
'\n'.join(rows))
def format_eval_html(exc_data, base_path, counter):
short_formatter = EvalHTMLFormatter(
base_path=base_path,
counter=counter,
include_reusable=False)
short_er = short_formatter.format_collected_data(exc_data)
long_formatter = EvalHTMLFormatter(
base_path=base_path,
counter=counter,
show_hidden_frames=True,
show_extra_data=False,
include_reusable=False)
long_er = long_formatter.format_collected_data(exc_data)
text_er = formatter.format_text(exc_data, show_hidden_frames=True)
if short_formatter.filter_frames(exc_data.frames) != \
long_formatter.filter_frames(exc_data.frames):
# Only display the full traceback when it differs from the
# short version
full_traceback_html = """
<br>
<script type="text/javascript">
show_button('full_traceback', 'full traceback')
</script>
<div id="full_traceback" class="hidden-data">
%s
</div>
""" % long_er
else:
full_traceback_html = ''
return """
%s
%s
<br>
<script type="text/javascript">
show_button('text_version', 'text version')
</script>
<div id="text_version" class="hidden-data">
<textarea style="width: 100%%" rows=10 cols=60>%s</textarea>
</div>
""" % (short_er, full_traceback_html, cgi.escape(text_er))
def make_repost_button(environ):
url = request.construct_url(environ)
if environ['REQUEST_METHOD'] == 'GET':
return ('<button onclick="window.location.href=%r">'
'Re-GET Page</button><br>' % url)
else:
# @@: I'd like to reconstruct this, but I can't because
# the POST body is probably lost at this point, and
# I can't get it back :(
return None
# @@: Use or lose the following code block
"""
fields = []
for name, value in wsgilib.parse_formvars(
environ, include_get_vars=False).items():
if hasattr(value, 'filename'):
# @@: Arg, we'll just submit the body, and leave out
# the filename :(
value = value.value
fields.append(
'<input type="hidden" name="%s" value="%s">'
% (html_quote(name), html_quote(value)))
return '''
<form action="%s" method="POST">
%s
<input type="submit" value="Re-POST Page">
</form>''' % (url, '\n'.join(fields))
"""
def input_form(tbid, debug_info):
return '''
<form action="#" method="POST"
onsubmit="return submitInput($(\'submit_%(tbid)s\'), %(tbid)s)">
<div id="exec-output-%(tbid)s" style="width: 95%%;
padding: 5px; margin: 5px; border: 2px solid #000;
display: none"></div>
<input type="text" name="input" id="debug_input_%(tbid)s"
style="width: 100%%"
autocomplete="off" onkeypress="upArrow(this, event)"><br>
<input type="submit" value="Execute" name="submitbutton"
onclick="return submitInput(this, %(tbid)s)"
id="submit_%(tbid)s"
input-from="debug_input_%(tbid)s"
output-to="exec-output-%(tbid)s">
<input type="submit" value="Expand"
onclick="return expandInput(this)">
</form>
''' % {'tbid': tbid}
error_template = '''
<html>
<head>
<title>Server Error</title>
%(head_html)s
</head>
<body>
<div id="error-area" style="display: none; background-color: #600; color: #fff; border: 2px solid black">
<div id="error-container"></div>
<button onclick="return clearError()">clear this</button>
</div>
%(repost_button)s
%(body)s
</body>
</html>
'''
def make_eval_exception(app, global_conf, xmlhttp_key=None):
"""
Wraps the application in an interactive debugger.
This debugger is a major security hole, and should only be
used during development.
xmlhttp_key is a string that, if present in QUERY_STRING,
indicates that the request is an XMLHttp request, and the
Javascript/interactive debugger should not be returned. (If you
try to put the debugger somewhere with innerHTML, you will often
crash the browser)
"""
if xmlhttp_key is None:
xmlhttp_key = global_conf.get('xmlhttp_key', '_')
return EvalException(app, xmlhttp_key=xmlhttp_key)

View File

@@ -0,0 +1,6 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
"""
Package for catching exceptions and displaying annotated exception
reports
"""

View File

@@ -0,0 +1,526 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
##############################################################################
#
# Copyright (c) 2001, 2002 Zope Corporation and Contributors.
# All Rights Reserved.
#
# This software is subject to the provisions of the Zope Public License,
# Version 2.0 (ZPL). A copy of the ZPL should accompany this distribution.
# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
# FOR A PARTICULAR PURPOSE.
#
##############################################################################
## Originally zExceptions.ExceptionFormatter from Zope;
## Modified by Ian Bicking, Imaginary Landscape, 2005
"""
An exception collector that finds traceback information plus
supplements
"""
import sys
import traceback
import time
try:
from cStringIO import StringIO
except ImportError:
from StringIO import StringIO
import linecache
from paste.exceptions import serial_number_generator
import warnings
DEBUG_EXCEPTION_FORMATTER = True
DEBUG_IDENT_PREFIX = 'E-'
FALLBACK_ENCODING = 'UTF-8'
__all__ = ['collect_exception', 'ExceptionCollector']
class ExceptionCollector(object):
"""
Produces a data structure that can be used by formatters to
display exception reports.
Magic variables:
If you define one of these variables in your local scope, you can
add information to tracebacks that happen in that context. This
allows applications to add all sorts of extra information about
the context of the error, including URLs, environmental variables,
users, hostnames, etc. These are the variables we look for:
``__traceback_supplement__``:
You can define this locally or globally (unlike all the other
variables, which must be defined locally).
``__traceback_supplement__`` is a tuple of ``(factory, arg1,
arg2...)``. When there is an exception, ``factory(arg1, arg2,
...)`` is called, and the resulting object is inspected for
supplemental information.
``__traceback_info__``:
This information is added to the traceback, usually fairly
literally.
``__traceback_hide__``:
If set and true, this indicates that the frame should be
hidden from abbreviated tracebacks. This way you can hide
some of the complexity of the larger framework and let the
user focus on their own errors.
By setting it to ``'before'``, all frames before this one will
be thrown away. By setting it to ``'after'`` then all frames
after this will be thrown away until ``'reset'`` is found. In
each case the frame where it is set is included, unless you
append ``'_and_this'`` to the value (e.g.,
``'before_and_this'``).
Note that formatters will ignore this entirely if the frame
that contains the error wouldn't normally be shown according
to these rules.
``__traceback_reporter__``:
This should be a reporter object (see the reporter module),
or a list/tuple of reporter objects. All reporters found this
way will be given the exception, innermost first.
``__traceback_decorator__``:
This object (defined in a local or global scope) will get the
result of this function (the CollectedException defined
below). It may modify this object in place, or return an
entirely new object. This gives the object the ability to
manipulate the traceback arbitrarily.
The actually interpretation of these values is largely up to the
reporters and formatters.
``collect_exception(*sys.exc_info())`` will return an object with
several attributes:
``frames``:
A list of frames
``exception_formatted``:
The formatted exception, generally a full traceback
``exception_type``:
The type of the exception, like ``ValueError``
``exception_value``:
The string value of the exception, like ``'x not in list'``
``identification_code``:
A hash of the exception data meant to identify the general
exception, so that it shares this code with other exceptions
that derive from the same problem. The code is a hash of
all the module names and function names in the traceback,
plus exception_type. This should be shown to users so they
can refer to the exception later. (@@: should it include a
portion that allows identification of the specific instance
of the exception as well?)
The list of frames goes innermost first. Each frame has these
attributes; some values may be None if they could not be
determined.
``modname``:
the name of the module
``filename``:
the filename of the module
``lineno``:
the line of the error
``revision``:
the contents of __version__ or __revision__
``name``:
the function name
``supplement``:
an object created from ``__traceback_supplement__``
``supplement_exception``:
a simple traceback of any exception ``__traceback_supplement__``
created
``traceback_info``:
the str() of any ``__traceback_info__`` variable found in the local
scope (@@: should it str()-ify it or not?)
``traceback_hide``:
the value of any ``__traceback_hide__`` variable
``traceback_log``:
the value of any ``__traceback_log__`` variable
``__traceback_supplement__`` is thrown away, but a fixed
set of attributes are captured; each of these attributes is
optional.
``object``:
the name of the object being visited
``source_url``:
the original URL requested
``line``:
the line of source being executed (for interpreters, like ZPT)
``column``:
the column of source being executed
``expression``:
the expression being evaluated (also for interpreters)
``warnings``:
a list of (string) warnings to be displayed
``getInfo``:
a function/method that takes no arguments, and returns a string
describing any extra information
``extraData``:
a function/method that takes no arguments, and returns a
dictionary. The contents of this dictionary will not be
displayed in the context of the traceback, but globally for
the exception. Results will be grouped by the keys in the
dictionaries (which also serve as titles). The keys can also
be tuples of (importance, title); in this case the importance
should be ``important`` (shows up at top), ``normal`` (shows
up somewhere; unspecified), ``supplemental`` (shows up at
bottom), or ``extra`` (shows up hidden or not at all).
These are used to create an object with attributes of the same
names (``getInfo`` becomes a string attribute, not a method).
``__traceback_supplement__`` implementations should be careful to
produce values that are relatively static and unlikely to cause
further errors in the reporting system -- any complex
introspection should go in ``getInfo()`` and should ultimately
return a string.
Note that all attributes are optional, and under certain
circumstances may be None or may not exist at all -- the collector
can only do a best effort, but must avoid creating any exceptions
itself.
Formatters may want to use ``__traceback_hide__`` as a hint to
hide frames that are part of the 'framework' or underlying system.
There are a variety of rules about special values for this
variables that formatters should be aware of.
TODO:
More attributes in __traceback_supplement__? Maybe an attribute
that gives a list of local variables that should also be
collected? Also, attributes that would be explicitly meant for
the entire request, not just a single frame. Right now some of
the fixed set of attributes (e.g., source_url) are meant for this
use, but there's no explicit way for the supplement to indicate
new values, e.g., logged-in user, HTTP referrer, environment, etc.
Also, the attributes that do exist are Zope/Web oriented.
More information on frames? cgitb, for instance, produces
extensive information on local variables. There exists the
possibility that getting this information may cause side effects,
which can make debugging more difficult; but it also provides
fodder for post-mortem debugging. However, the collector is not
meant to be configurable, but to capture everything it can and let
the formatters be configurable. Maybe this would have to be a
configuration value, or maybe it could be indicated by another
magical variable (which would probably mean 'show all local
variables below this frame')
"""
show_revisions = 0
def __init__(self, limit=None):
self.limit = limit
def getLimit(self):
limit = self.limit
if limit is None:
limit = getattr(sys, 'tracebacklimit', None)
return limit
def getRevision(self, globals):
if not self.show_revisions:
return None
revision = globals.get('__revision__', None)
if revision is None:
# Incorrect but commonly used spelling
revision = globals.get('__version__', None)
if revision is not None:
try:
revision = str(revision).strip()
except:
revision = '???'
return revision
def collectSupplement(self, supplement, tb):
result = {}
for name in ('object', 'source_url', 'line', 'column',
'expression', 'warnings'):
result[name] = getattr(supplement, name, None)
func = getattr(supplement, 'getInfo', None)
if func:
result['info'] = func()
else:
result['info'] = None
func = getattr(supplement, 'extraData', None)
if func:
result['extra'] = func()
else:
result['extra'] = None
return SupplementaryData(**result)
def collectLine(self, tb, extra_data):
f = tb.tb_frame
lineno = tb.tb_lineno
co = f.f_code
filename = co.co_filename
name = co.co_name
globals = f.f_globals
locals = f.f_locals
if not hasattr(locals, 'has_key'):
# Something weird about this frame; it's not a real dict
warnings.warn(
"Frame %s has an invalid locals(): %r" % (
globals.get('__name__', 'unknown'), locals))
locals = {}
data = {}
data['modname'] = globals.get('__name__', None)
data['filename'] = filename
data['lineno'] = lineno
data['revision'] = self.getRevision(globals)
data['name'] = name
data['tbid'] = id(tb)
# Output a traceback supplement, if any.
if locals.has_key('__traceback_supplement__'):
# Use the supplement defined in the function.
tbs = locals['__traceback_supplement__']
elif globals.has_key('__traceback_supplement__'):
# Use the supplement defined in the module.
# This is used by Scripts (Python).
tbs = globals['__traceback_supplement__']
else:
tbs = None
if tbs is not None:
factory = tbs[0]
args = tbs[1:]
try:
supp = factory(*args)
data['supplement'] = self.collectSupplement(supp, tb)
if data['supplement'].extra:
for key, value in data['supplement'].extra.items():
extra_data.setdefault(key, []).append(value)
except:
if DEBUG_EXCEPTION_FORMATTER:
out = StringIO()
traceback.print_exc(file=out)
text = out.getvalue()
data['supplement_exception'] = text
# else just swallow the exception.
try:
tbi = locals.get('__traceback_info__', None)
if tbi is not None:
data['traceback_info'] = str(tbi)
except:
pass
marker = []
for name in ('__traceback_hide__', '__traceback_log__',
'__traceback_decorator__'):
try:
tbh = locals.get(name, globals.get(name, marker))
if tbh is not marker:
data[name[2:-2]] = tbh
except:
pass
return data
def collectExceptionOnly(self, etype, value):
return traceback.format_exception_only(etype, value)
def collectException(self, etype, value, tb, limit=None):
# The next line provides a way to detect recursion.
__exception_formatter__ = 1
frames = []
ident_data = []
traceback_decorators = []
if limit is None:
limit = self.getLimit()
n = 0
extra_data = {}
while tb is not None and (limit is None or n < limit):
if tb.tb_frame.f_locals.get('__exception_formatter__'):
# Stop recursion. @@: should make a fake ExceptionFrame
frames.append('(Recursive formatException() stopped)\n')
break
data = self.collectLine(tb, extra_data)
frame = ExceptionFrame(**data)
frames.append(frame)
if frame.traceback_decorator is not None:
traceback_decorators.append(frame.traceback_decorator)
ident_data.append(frame.modname or '?')
ident_data.append(frame.name or '?')
tb = tb.tb_next
n = n + 1
ident_data.append(str(etype))
ident = serial_number_generator.hash_identifier(
' '.join(ident_data), length=5, upper=True,
prefix=DEBUG_IDENT_PREFIX)
result = CollectedException(
frames=frames,
exception_formatted=self.collectExceptionOnly(etype, value),
exception_type=etype,
exception_value=self.safeStr(value),
identification_code=ident,
date=time.localtime(),
extra_data=extra_data)
if etype is ImportError:
extra_data[('important', 'sys.path')] = [sys.path]
for decorator in traceback_decorators:
try:
new_result = decorator(result)
if new_result is not None:
result = new_result
except:
pass
return result
def safeStr(self, obj):
try:
return str(obj)
except UnicodeEncodeError:
try:
return unicode(obj).encode(FALLBACK_ENCODING, 'replace')
except UnicodeEncodeError:
# This is when something is really messed up, but this can
# happen when the __str__ of an object has to handle unicode
return repr(obj)
limit = 200
class Bunch(object):
"""
A generic container
"""
def __init__(self, **attrs):
for name, value in attrs.items():
setattr(self, name, value)
def __repr__(self):
name = '<%s ' % self.__class__.__name__
name += ' '.join(['%s=%r' % (name, str(value)[:30])
for name, value in self.__dict__.items()
if not name.startswith('_')])
return name + '>'
class CollectedException(Bunch):
"""
This is the result of collection the exception; it contains copies
of data of interest.
"""
# A list of frames (ExceptionFrame instances), innermost last:
frames = []
# The result of traceback.format_exception_only; this looks
# like a normal traceback you'd see in the interactive interpreter
exception_formatted = None
# The *string* representation of the type of the exception
# (@@: should we give the # actual class? -- we can't keep the
# actual exception around, but the class should be safe)
# Something like 'ValueError'
exception_type = None
# The string representation of the exception, from ``str(e)``.
exception_value = None
# An identifier which should more-or-less classify this particular
# exception, including where in the code it happened.
identification_code = None
# The date, as time.localtime() returns:
date = None
# A dictionary of supplemental data:
extra_data = {}
class SupplementaryData(Bunch):
"""
The result of __traceback_supplement__. We don't keep the
supplement object around, for fear of GC problems and whatnot.
(@@: Maybe I'm being too superstitious about copying only specific
information over)
"""
# These attributes are copied from the object, or left as None
# if the object doesn't have these attributes:
object = None
source_url = None
line = None
column = None
expression = None
warnings = None
# This is the *return value* of supplement.getInfo():
info = None
class ExceptionFrame(Bunch):
"""
This represents one frame of the exception. Each frame is a
context in the call stack, typically represented by a line
number and module name in the traceback.
"""
# The name of the module; can be None, especially when the code
# isn't associated with a module.
modname = None
# The filename (@@: when no filename, is it None or '?'?)
filename = None
# Line number
lineno = None
# The value of __revision__ or __version__ -- but only if
# show_revision = True (by defaut it is false). (@@: Why not
# collect this?)
revision = None
# The name of the function with the error (@@: None or '?' when
# unknown?)
name = None
# A SupplementaryData object, if __traceback_supplement__ was found
# (and produced no errors)
supplement = None
# If accessing __traceback_supplement__ causes any error, the
# plain-text traceback is stored here
supplement_exception = None
# The str() of any __traceback_info__ value found
traceback_info = None
# The value of __traceback_hide__
traceback_hide = False
# The value of __traceback_decorator__
traceback_decorator = None
# The id() of the traceback scope, can be used to reference the
# scope for use elsewhere
tbid = None
def get_source_line(self, context=0):
"""
Return the source of the current line of this frame. You
probably want to .strip() it as well, as it is likely to have
leading whitespace.
If context is given, then that many lines on either side will
also be returned. E.g., context=1 will give 3 lines.
"""
if not self.filename or not self.lineno:
return None
lines = []
for lineno in range(self.lineno-context, self.lineno+context+1):
lines.append(linecache.getline(self.filename, lineno))
return ''.join(lines)
if hasattr(sys, 'tracebacklimit'):
limit = min(limit, sys.tracebacklimit)
col = ExceptionCollector()
def collect_exception(t, v, tb, limit=None):
"""
Collection an exception from ``sys.exc_info()``.
Use like::
try:
blah blah
except:
exc_data = collect_exception(*sys.exc_info())
"""
return col.collectException(t, v, tb, limit=limit)

View File

@@ -0,0 +1,460 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
"""
Error handler middleware
"""
import sys
import traceback
import cgi
try:
from cStringIO import StringIO
except ImportError:
from StringIO import StringIO
from paste.exceptions import formatter, collector, reporter
from paste import wsgilib
from paste import request
__all__ = ['ErrorMiddleware', 'handle_exception']
class _NoDefault(object):
def __repr__(self):
return '<NoDefault>'
NoDefault = _NoDefault()
class ErrorMiddleware(object):
"""
Error handling middleware
Usage::
error_catching_wsgi_app = ErrorMiddleware(wsgi_app)
Settings:
``debug``:
If true, then tracebacks will be shown in the browser.
``error_email``:
an email address (or list of addresses) to send exception
reports to
``error_log``:
a filename to append tracebacks to
``show_exceptions_in_wsgi_errors``:
If true, then errors will be printed to ``wsgi.errors``
(frequently a server error log, or stderr).
``from_address``, ``smtp_server``, ``error_subject_prefix``, ``smtp_username``, ``smtp_password``, ``smtp_use_tls``:
variables to control the emailed exception reports
``error_message``:
When debug mode is off, the error message to show to users.
``xmlhttp_key``:
When this key (default ``_``) is in the request GET variables
(not POST!), expect that this is an XMLHttpRequest, and the
response should be more minimal; it should not be a complete
HTML page.
Environment Configuration:
``paste.throw_errors``:
If this setting in the request environment is true, then this
middleware is disabled. This can be useful in a testing situation
where you don't want errors to be caught and transformed.
``paste.expected_exceptions``:
When this middleware encounters an exception listed in this
environment variable and when the ``start_response`` has not
yet occurred, the exception will be re-raised instead of being
caught. This should generally be set by middleware that may
(but probably shouldn't be) installed above this middleware,
and wants to get certain exceptions. Exceptions raised after
``start_response`` have been called are always caught since
by definition they are no longer expected.
"""
def __init__(self, application, global_conf=None,
debug=NoDefault,
error_email=None,
error_log=None,
show_exceptions_in_wsgi_errors=NoDefault,
from_address=None,
smtp_server=None,
smtp_username=None,
smtp_password=None,
smtp_use_tls=False,
error_subject_prefix=None,
error_message=None,
xmlhttp_key=None):
from paste.util import converters
self.application = application
# @@: global_conf should be handled elsewhere in a separate
# function for the entry point
if global_conf is None:
global_conf = {}
if debug is NoDefault:
debug = converters.asbool(global_conf.get('debug'))
if show_exceptions_in_wsgi_errors is NoDefault:
show_exceptions_in_wsgi_errors = converters.asbool(global_conf.get('show_exceptions_in_wsgi_errors'))
self.debug_mode = converters.asbool(debug)
if error_email is None:
error_email = (global_conf.get('error_email')
or global_conf.get('admin_email')
or global_conf.get('webmaster_email')
or global_conf.get('sysadmin_email'))
self.error_email = converters.aslist(error_email)
self.error_log = error_log
self.show_exceptions_in_wsgi_errors = show_exceptions_in_wsgi_errors
if from_address is None:
from_address = global_conf.get('error_from_address', 'errors@localhost')
self.from_address = from_address
if smtp_server is None:
smtp_server = global_conf.get('smtp_server', 'localhost')
self.smtp_server = smtp_server
self.smtp_username = smtp_username or global_conf.get('smtp_username')
self.smtp_password = smtp_password or global_conf.get('smtp_password')
self.smtp_use_tls = smtp_use_tls or converters.asbool(global_conf.get('smtp_use_tls'))
self.error_subject_prefix = error_subject_prefix or ''
if error_message is None:
error_message = global_conf.get('error_message')
self.error_message = error_message
if xmlhttp_key is None:
xmlhttp_key = global_conf.get('xmlhttp_key', '_')
self.xmlhttp_key = xmlhttp_key
def __call__(self, environ, start_response):
"""
The WSGI application interface.
"""
# We want to be careful about not sending headers twice,
# and the content type that the app has committed to (if there
# is an exception in the iterator body of the response)
if environ.get('paste.throw_errors'):
return self.application(environ, start_response)
environ['paste.throw_errors'] = True
try:
__traceback_supplement__ = Supplement, self, environ
sr_checker = ResponseStartChecker(start_response)
app_iter = self.application(environ, sr_checker)
return self.make_catching_iter(app_iter, environ, sr_checker)
except:
exc_info = sys.exc_info()
try:
for expect in environ.get('paste.expected_exceptions', []):
if isinstance(exc_info[1], expect):
raise
start_response('500 Internal Server Error',
[('content-type', 'text/html')],
exc_info)
# @@: it would be nice to deal with bad content types here
response = self.exception_handler(exc_info, environ)
return [response]
finally:
# clean up locals...
exc_info = None
def make_catching_iter(self, app_iter, environ, sr_checker):
if isinstance(app_iter, (list, tuple)):
# These don't raise
return app_iter
return CatchingIter(app_iter, environ, sr_checker, self)
def exception_handler(self, exc_info, environ):
simple_html_error = False
if self.xmlhttp_key:
get_vars = wsgilib.parse_querystring(environ)
if dict(get_vars).get(self.xmlhttp_key):
simple_html_error = True
return handle_exception(
exc_info, environ['wsgi.errors'],
html=True,
debug_mode=self.debug_mode,
error_email=self.error_email,
error_log=self.error_log,
show_exceptions_in_wsgi_errors=self.show_exceptions_in_wsgi_errors,
error_email_from=self.from_address,
smtp_server=self.smtp_server,
smtp_username=self.smtp_username,
smtp_password=self.smtp_password,
smtp_use_tls=self.smtp_use_tls,
error_subject_prefix=self.error_subject_prefix,
error_message=self.error_message,
simple_html_error=simple_html_error)
class ResponseStartChecker(object):
def __init__(self, start_response):
self.start_response = start_response
self.response_started = False
def __call__(self, *args):
self.response_started = True
self.start_response(*args)
class CatchingIter(object):
"""
A wrapper around the application iterator that will catch
exceptions raised by the a generator, or by the close method, and
display or report as necessary.
"""
def __init__(self, app_iter, environ, start_checker, error_middleware):
self.app_iterable = app_iter
self.app_iterator = iter(app_iter)
self.environ = environ
self.start_checker = start_checker
self.error_middleware = error_middleware
self.closed = False
def __iter__(self):
return self
def next(self):
__traceback_supplement__ = (
Supplement, self.error_middleware, self.environ)
if self.closed:
raise StopIteration
try:
return self.app_iterator.next()
except StopIteration:
self.closed = True
close_response = self._close()
if close_response is not None:
return close_response
else:
raise StopIteration
except:
self.closed = True
close_response = self._close()
exc_info = sys.exc_info()
response = self.error_middleware.exception_handler(
exc_info, self.environ)
if close_response is not None:
response += (
'<hr noshade>Error in .close():<br>%s'
% close_response)
if not self.start_checker.response_started:
self.start_checker('500 Internal Server Error',
[('content-type', 'text/html')],
exc_info)
return response
def close(self):
# This should at least print something to stderr if the
# close method fails at this point
if not self.closed:
self._close()
def _close(self):
"""Close and return any error message"""
if not hasattr(self.app_iterable, 'close'):
return None
try:
self.app_iterable.close()
return None
except:
close_response = self.error_middleware.exception_handler(
sys.exc_info(), self.environ)
return close_response
class Supplement(object):
"""
This is a supplement used to display standard WSGI information in
the traceback.
"""
def __init__(self, middleware, environ):
self.middleware = middleware
self.environ = environ
self.source_url = request.construct_url(environ)
def extraData(self):
data = {}
cgi_vars = data[('extra', 'CGI Variables')] = {}
wsgi_vars = data[('extra', 'WSGI Variables')] = {}
hide_vars = ['paste.config', 'wsgi.errors', 'wsgi.input',
'wsgi.multithread', 'wsgi.multiprocess',
'wsgi.run_once', 'wsgi.version',
'wsgi.url_scheme']
for name, value in self.environ.items():
if name.upper() == name:
if value:
cgi_vars[name] = value
elif name not in hide_vars:
wsgi_vars[name] = value
if self.environ['wsgi.version'] != (1, 0):
wsgi_vars['wsgi.version'] = self.environ['wsgi.version']
proc_desc = tuple([int(bool(self.environ[key]))
for key in ('wsgi.multiprocess',
'wsgi.multithread',
'wsgi.run_once')])
wsgi_vars['wsgi process'] = self.process_combos[proc_desc]
wsgi_vars['application'] = self.middleware.application
if 'paste.config' in self.environ:
data[('extra', 'Configuration')] = dict(self.environ['paste.config'])
return data
process_combos = {
# multiprocess, multithread, run_once
(0, 0, 0): 'Non-concurrent server',
(0, 1, 0): 'Multithreaded',
(1, 0, 0): 'Multiprocess',
(1, 1, 0): 'Multi process AND threads (?)',
(0, 0, 1): 'Non-concurrent CGI',
(0, 1, 1): 'Multithread CGI (?)',
(1, 0, 1): 'CGI',
(1, 1, 1): 'Multi thread/process CGI (?)',
}
def handle_exception(exc_info, error_stream, html=True,
debug_mode=False,
error_email=None,
error_log=None,
show_exceptions_in_wsgi_errors=False,
error_email_from='errors@localhost',
smtp_server='localhost',
smtp_username=None,
smtp_password=None,
smtp_use_tls=False,
error_subject_prefix='',
error_message=None,
simple_html_error=False,
):
"""
For exception handling outside of a web context
Use like::
import sys
from paste.exceptions.errormiddleware import handle_exception
try:
do stuff
except:
handle_exception(
sys.exc_info(), sys.stderr, html=False, ...other config...)
If you want to report, but not fully catch the exception, call
``raise`` after ``handle_exception``, which (when given no argument)
will reraise the exception.
"""
reported = False
exc_data = collector.collect_exception(*exc_info)
extra_data = ''
if error_email:
rep = reporter.EmailReporter(
to_addresses=error_email,
from_address=error_email_from,
smtp_server=smtp_server,
smtp_username=smtp_username,
smtp_password=smtp_password,
smtp_use_tls=smtp_use_tls,
subject_prefix=error_subject_prefix)
rep_err = send_report(rep, exc_data, html=html)
if rep_err:
extra_data += rep_err
else:
reported = True
if error_log:
rep = reporter.LogReporter(
filename=error_log)
rep_err = send_report(rep, exc_data, html=html)
if rep_err:
extra_data += rep_err
else:
reported = True
if show_exceptions_in_wsgi_errors:
rep = reporter.FileReporter(
file=error_stream)
rep_err = send_report(rep, exc_data, html=html)
if rep_err:
extra_data += rep_err
else:
reported = True
else:
error_stream.write('Error - %s: %s\n' % (
exc_data.exception_type, exc_data.exception_value))
if html:
if debug_mode and simple_html_error:
return_error = formatter.format_html(
exc_data, include_hidden_frames=False,
include_reusable=False, show_extra_data=False)
reported = True
elif debug_mode and not simple_html_error:
error_html = formatter.format_html(
exc_data,
include_hidden_frames=True,
include_reusable=False)
head_html = formatter.error_css + formatter.hide_display_js
return_error = error_template(
head_html, error_html, extra_data)
extra_data = ''
reported = True
else:
msg = error_message or '''
An error occurred. See the error logs for more information.
(Turn debug on to display exception reports here)
'''
return_error = error_template('', msg, '')
else:
return_error = None
if not reported and error_stream:
err_report = formatter.format_text(exc_data, show_hidden_frames=True)
err_report += '\n' + '-'*60 + '\n'
error_stream.write(err_report)
if extra_data:
error_stream.write(extra_data)
return return_error
def send_report(rep, exc_data, html=True):
try:
rep.report(exc_data)
except:
output = StringIO()
traceback.print_exc(file=output)
if html:
return """
<p>Additionally an error occurred while sending the %s report:
<pre>%s</pre>
</p>""" % (
cgi.escape(str(rep)), output.getvalue())
else:
return (
"Additionally an error occurred while sending the "
"%s report:\n%s" % (str(rep), output.getvalue()))
else:
return ''
def error_template(head_html, exception, extra):
return '''
<html>
<head>
<title>Server Error</title>
%s
</head>
<body>
<h1>Server Error</h1>
%s
%s
</body>
</html>''' % (head_html, exception, extra)
def make_error_middleware(app, global_conf, **kw):
return ErrorMiddleware(app, global_conf=global_conf, **kw)
doc_lines = ErrorMiddleware.__doc__.splitlines(True)
for i in range(len(doc_lines)):
if doc_lines[i].strip().startswith('Settings'):
make_error_middleware.__doc__ = ''.join(doc_lines[i:])
break
del i, doc_lines

View File

@@ -0,0 +1,564 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
"""
Formatters for the exception data that comes from ExceptionCollector.
"""
# @@: TODO:
# Use this: http://www.zope.org/Members/tino/VisualTraceback/VisualTracebackNews
import cgi
import re
from paste.util import PySourceColor
def html_quote(s):
return cgi.escape(str(s), True)
class AbstractFormatter(object):
general_data_order = ['object', 'source_url']
def __init__(self, show_hidden_frames=False,
include_reusable=True,
show_extra_data=True,
trim_source_paths=()):
self.show_hidden_frames = show_hidden_frames
self.trim_source_paths = trim_source_paths
self.include_reusable = include_reusable
self.show_extra_data = show_extra_data
def format_collected_data(self, exc_data):
general_data = {}
if self.show_extra_data:
for name, value_list in exc_data.extra_data.items():
if isinstance(name, tuple):
importance, title = name
else:
importance, title = 'normal', name
for value in value_list:
general_data[(importance, name)] = self.format_extra_data(
importance, title, value)
lines = []
frames = self.filter_frames(exc_data.frames)
for frame in frames:
sup = frame.supplement
if sup:
if sup.object:
general_data[('important', 'object')] = self.format_sup_object(
sup.object)
if sup.source_url:
general_data[('important', 'source_url')] = self.format_sup_url(
sup.source_url)
if sup.line:
lines.append(self.format_sup_line_pos(sup.line, sup.column))
if sup.expression:
lines.append(self.format_sup_expression(sup.expression))
if sup.warnings:
for warning in sup.warnings:
lines.append(self.format_sup_warning(warning))
if sup.info:
lines.extend(self.format_sup_info(sup.info))
if frame.supplement_exception:
lines.append('Exception in supplement:')
lines.append(self.quote_long(frame.supplement_exception))
if frame.traceback_info:
lines.append(self.format_traceback_info(frame.traceback_info))
filename = frame.filename
if filename and self.trim_source_paths:
for path, repl in self.trim_source_paths:
if filename.startswith(path):
filename = repl + filename[len(path):]
break
lines.append(self.format_source_line(filename or '?', frame))
source = frame.get_source_line()
long_source = frame.get_source_line(2)
if source:
lines.append(self.format_long_source(
source, long_source))
etype = exc_data.exception_type
if not isinstance(etype, basestring):
etype = etype.__name__
exc_info = self.format_exception_info(
etype,
exc_data.exception_value)
data_by_importance = {'important': [], 'normal': [],
'supplemental': [], 'extra': []}
for (importance, name), value in general_data.items():
data_by_importance[importance].append(
(name, value))
for value in data_by_importance.values():
value.sort()
return self.format_combine(data_by_importance, lines, exc_info)
def filter_frames(self, frames):
"""
Removes any frames that should be hidden, according to the
values of traceback_hide, self.show_hidden_frames, and the
hidden status of the final frame.
"""
if self.show_hidden_frames:
return frames
new_frames = []
hidden = False
for frame in frames:
hide = frame.traceback_hide
# @@: It would be nice to signal a warning if an unknown
# hide string was used, but I'm not sure where to put
# that warning.
if hide == 'before':
new_frames = []
hidden = False
elif hide == 'before_and_this':
new_frames = []
hidden = False
continue
elif hide == 'reset':
hidden = False
elif hide == 'reset_and_this':
hidden = False
continue
elif hide == 'after':
hidden = True
elif hide == 'after_and_this':
hidden = True
continue
elif hide:
continue
elif hidden:
continue
new_frames.append(frame)
if frames[-1] not in new_frames:
# We must include the last frame; that we don't indicates
# that the error happened where something was "hidden",
# so we just have to show everything
return frames
return new_frames
def pretty_string_repr(self, s):
"""
Formats the string as a triple-quoted string when it contains
newlines.
"""
if '\n' in s:
s = repr(s)
s = s[0]*3 + s[1:-1] + s[-1]*3
s = s.replace('\\n', '\n')
return s
else:
return repr(s)
def long_item_list(self, lst):
"""
Returns true if the list contains items that are long, and should
be more nicely formatted.
"""
how_many = 0
for item in lst:
if len(repr(item)) > 40:
how_many += 1
if how_many >= 3:
return True
return False
class TextFormatter(AbstractFormatter):
def quote(self, s):
return s
def quote_long(self, s):
return s
def emphasize(self, s):
return s
def format_sup_object(self, obj):
return 'In object: %s' % self.emphasize(self.quote(repr(obj)))
def format_sup_url(self, url):
return 'URL: %s' % self.quote(url)
def format_sup_line_pos(self, line, column):
if column:
return self.emphasize('Line %i, Column %i' % (line, column))
else:
return self.emphasize('Line %i' % line)
def format_sup_expression(self, expr):
return self.emphasize('In expression: %s' % self.quote(expr))
def format_sup_warning(self, warning):
return 'Warning: %s' % self.quote(warning)
def format_sup_info(self, info):
return [self.quote_long(info)]
def format_source_line(self, filename, frame):
return 'File %r, line %s in %s' % (
filename, frame.lineno or '?', frame.name or '?')
def format_long_source(self, source, long_source):
return self.format_source(source)
def format_source(self, source_line):
return ' ' + self.quote(source_line.strip())
def format_exception_info(self, etype, evalue):
return self.emphasize(
'%s: %s' % (self.quote(etype), self.quote(evalue)))
def format_traceback_info(self, info):
return info
def format_combine(self, data_by_importance, lines, exc_info):
lines[:0] = [value for n, value in data_by_importance['important']]
lines.append(exc_info)
for name in 'normal', 'supplemental', 'extra':
lines.extend([value for n, value in data_by_importance[name]])
return self.format_combine_lines(lines)
def format_combine_lines(self, lines):
return '\n'.join(lines)
def format_extra_data(self, importance, title, value):
if isinstance(value, str):
s = self.pretty_string_repr(value)
if '\n' in s:
return '%s:\n%s' % (title, s)
else:
return '%s: %s' % (title, s)
elif isinstance(value, dict):
lines = ['\n', title, '-'*len(title)]
items = value.items()
items.sort()
for n, v in items:
try:
v = repr(v)
except Exception, e:
v = 'Cannot display: %s' % e
v = truncate(v)
lines.append(' %s: %s' % (n, v))
return '\n'.join(lines)
elif (isinstance(value, (list, tuple))
and self.long_item_list(value)):
parts = [truncate(repr(v)) for v in value]
return '%s: [\n %s]' % (
title, ',\n '.join(parts))
else:
return '%s: %s' % (title, truncate(repr(value)))
class HTMLFormatter(TextFormatter):
def quote(self, s):
return html_quote(s)
def quote_long(self, s):
return '<pre>%s</pre>' % self.quote(s)
def emphasize(self, s):
return '<b>%s</b>' % s
def format_sup_url(self, url):
return 'URL: <a href="%s">%s</a>' % (url, url)
def format_combine_lines(self, lines):
return '<br>\n'.join(lines)
def format_source_line(self, filename, frame):
name = self.quote(frame.name or '?')
return 'Module <span class="module" title="%s">%s</span>:<b>%s</b> in <code>%s</code>' % (
filename, frame.modname or '?', frame.lineno or '?',
name)
return 'File %r, line %s in <tt>%s</tt>' % (
filename, frame.lineno, name)
def format_long_source(self, source, long_source):
q_long_source = str2html(long_source, False, 4, True)
q_source = str2html(source, True, 0, False)
return ('<code style="display: none" class="source" source-type="long"><a class="switch_source" onclick="return switch_source(this, \'long\')" href="#">&lt;&lt;&nbsp; </a>%s</code>'
'<code class="source" source-type="short"><a onclick="return switch_source(this, \'short\')" class="switch_source" href="#">&gt;&gt;&nbsp; </a>%s</code>'
% (q_long_source,
q_source))
def format_source(self, source_line):
return '&nbsp;&nbsp;<code class="source">%s</code>' % self.quote(source_line.strip())
def format_traceback_info(self, info):
return '<pre>%s</pre>' % self.quote(info)
def format_extra_data(self, importance, title, value):
if isinstance(value, str):
s = self.pretty_string_repr(value)
if '\n' in s:
return '%s:<br><pre>%s</pre>' % (title, self.quote(s))
else:
return '%s: <tt>%s</tt>' % (title, self.quote(s))
elif isinstance(value, dict):
return self.zebra_table(title, value)
elif (isinstance(value, (list, tuple))
and self.long_item_list(value)):
return '%s: <tt>[<br>\n&nbsp; &nbsp; %s]</tt>' % (
title, ',<br>&nbsp; &nbsp; '.join(map(self.quote, map(repr, value))))
else:
return '%s: <tt>%s</tt>' % (title, self.quote(repr(value)))
def format_combine(self, data_by_importance, lines, exc_info):
lines[:0] = [value for n, value in data_by_importance['important']]
lines.append(exc_info)
for name in 'normal', 'supplemental':
lines.extend([value for n, value in data_by_importance[name]])
if data_by_importance['extra']:
lines.append(
'<script type="text/javascript">\nshow_button(\'extra_data\', \'extra data\');\n</script>\n' +
'<div id="extra_data" class="hidden-data">\n')
lines.extend([value for n, value in data_by_importance['extra']])
lines.append('</div>')
text = self.format_combine_lines(lines)
if self.include_reusable:
return error_css + hide_display_js + text
else:
# Usually because another error is already on this page,
# and so the js & CSS are unneeded
return text
def zebra_table(self, title, rows, table_class="variables"):
if isinstance(rows, dict):
rows = rows.items()
rows.sort()
table = ['<table class="%s">' % table_class,
'<tr class="header"><th colspan="2">%s</th></tr>'
% self.quote(title)]
odd = False
for name, value in rows:
try:
value = repr(value)
except Exception, e:
value = 'Cannot print: %s' % e
odd = not odd
table.append(
'<tr class="%s"><td>%s</td>'
% (odd and 'odd' or 'even', self.quote(name)))
table.append(
'<td><tt>%s</tt></td></tr>'
% make_wrappable(self.quote(truncate(value))))
table.append('</table>')
return '\n'.join(table)
hide_display_js = r'''
<script type="text/javascript">
function hide_display(id) {
var el = document.getElementById(id);
if (el.className == "hidden-data") {
el.className = "";
return true;
} else {
el.className = "hidden-data";
return false;
}
}
document.write('<style type="text/css">\n');
document.write('.hidden-data {display: none}\n');
document.write('</style>\n');
function show_button(toggle_id, name) {
document.write('<a href="#' + toggle_id
+ '" onclick="javascript:hide_display(\'' + toggle_id
+ '\')" class="button">' + name + '</a><br>');
}
function switch_source(el, hide_type) {
while (el) {
if (el.getAttribute &&
el.getAttribute('source-type') == hide_type) {
break;
}
el = el.parentNode;
}
if (! el) {
return false;
}
el.style.display = 'none';
if (hide_type == 'long') {
while (el) {
if (el.getAttribute &&
el.getAttribute('source-type') == 'short') {
break;
}
el = el.nextSibling;
}
} else {
while (el) {
if (el.getAttribute &&
el.getAttribute('source-type') == 'long') {
break;
}
el = el.previousSibling;
}
}
if (el) {
el.style.display = '';
}
return false;
}
</script>'''
error_css = """
<style type="text/css">
body {
font-family: Helvetica, sans-serif;
}
table {
width: 100%;
}
tr.header {
background-color: #006;
color: #fff;
}
tr.even {
background-color: #ddd;
}
table.variables td {
vertical-align: top;
overflow: auto;
}
a.button {
background-color: #ccc;
border: 2px outset #aaa;
color: #000;
text-decoration: none;
}
a.button:hover {
background-color: #ddd;
}
code.source {
color: #006;
}
a.switch_source {
color: #090;
text-decoration: none;
}
a.switch_source:hover {
background-color: #ddd;
}
.source-highlight {
background-color: #ff9;
}
</style>
"""
def format_html(exc_data, include_hidden_frames=False, **ops):
if not include_hidden_frames:
return HTMLFormatter(**ops).format_collected_data(exc_data)
short_er = format_html(exc_data, show_hidden_frames=False, **ops)
# @@: This should have a way of seeing if the previous traceback
# was actually trimmed at all
ops['include_reusable'] = False
ops['show_extra_data'] = False
long_er = format_html(exc_data, show_hidden_frames=True, **ops)
text_er = format_text(exc_data, show_hidden_frames=True, **ops)
return """
%s
<br>
<script type="text/javascript">
show_button('full_traceback', 'full traceback')
</script>
<div id="full_traceback" class="hidden-data">
%s
</div>
<br>
<script type="text/javascript">
show_button('text_version', 'text version')
</script>
<div id="text_version" class="hidden-data">
<textarea style="width: 100%%" rows=10 cols=60>%s</textarea>
</div>
""" % (short_er, long_er, cgi.escape(text_er))
def format_text(exc_data, **ops):
return TextFormatter(**ops).format_collected_data(exc_data)
whitespace_re = re.compile(r' +')
pre_re = re.compile(r'</?pre.*?>')
error_re = re.compile(r'<h3>ERROR: .*?</h3>')
def str2html(src, strip=False, indent_subsequent=0,
highlight_inner=False):
"""
Convert a string to HTML. Try to be really safe about it,
returning a quoted version of the string if nothing else works.
"""
try:
return _str2html(src, strip=strip,
indent_subsequent=indent_subsequent,
highlight_inner=highlight_inner)
except:
return html_quote(src)
def _str2html(src, strip=False, indent_subsequent=0,
highlight_inner=False):
if strip:
src = src.strip()
orig_src = src
try:
src = PySourceColor.str2html(src, form='snip')
src = error_re.sub('', src)
src = pre_re.sub('', src)
src = re.sub(r'^[\n\r]{0,1}', '', src)
src = re.sub(r'[\n\r]{0,1}$', '', src)
except:
src = html_quote(orig_src)
lines = src.splitlines()
if len(lines) == 1:
return lines[0]
indent = ' '*indent_subsequent
for i in range(1, len(lines)):
lines[i] = indent+lines[i]
if highlight_inner and i == len(lines)/2:
lines[i] = '<span class="source-highlight">%s</span>' % lines[i]
src = '<br>\n'.join(lines)
src = whitespace_re.sub(
lambda m: '&nbsp;'*(len(m.group(0))-1) + ' ', src)
return src
def truncate(string, limit=1000):
"""
Truncate the string to the limit number of
characters
"""
if len(string) > limit:
return string[:limit-20]+'...'+string[-17:]
else:
return string
def make_wrappable(html, wrap_limit=60,
split_on=';?&@!$#-/\\"\''):
# Currently using <wbr>, maybe should use &#8203;
# http://www.cs.tut.fi/~jkorpela/html/nobr.html
if len(html) <= wrap_limit:
return html
words = html.split()
new_words = []
for word in words:
wrapped_word = ''
while len(word) > wrap_limit:
for char in split_on:
if char in word:
first, rest = word.split(char, 1)
wrapped_word += first+char+'<wbr>'
word = rest
break
else:
for i in range(0, len(word), wrap_limit):
wrapped_word += word[i:i+wrap_limit]+'<wbr>'
word = ''
wrapped_word += word
new_words.append(wrapped_word)
return ' '.join(new_words)
def make_pre_wrappable(html, wrap_limit=60,
split_on=';?&@!$#-/\\"\''):
"""
Like ``make_wrappable()`` but intended for text that will
go in a ``<pre>`` block, so wrap on a line-by-line basis.
"""
lines = html.splitlines()
new_lines = []
for line in lines:
if len(line) > wrap_limit:
for char in split_on:
if char in line:
parts = line.split(char)
line = '<wbr>'.join(parts)
break
new_lines.append(line)
return '\n'.join(lines)

View File

@@ -0,0 +1,141 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
from email.MIMEText import MIMEText
from email.MIMEMultipart import MIMEMultipart
import smtplib
import time
try:
from socket import sslerror
except ImportError:
sslerror = None
from paste.exceptions import formatter
class Reporter(object):
def __init__(self, **conf):
for name, value in conf.items():
if not hasattr(self, name):
raise TypeError(
"The keyword argument %s was not expected"
% name)
setattr(self, name, value)
self.check_params()
def check_params(self):
pass
def format_date(self, exc_data):
return time.strftime('%c', exc_data.date)
def format_html(self, exc_data, **kw):
return formatter.format_html(exc_data, **kw)
def format_text(self, exc_data, **kw):
return formatter.format_text(exc_data, **kw)
class EmailReporter(Reporter):
to_addresses = None
from_address = None
smtp_server = 'localhost'
smtp_username = None
smtp_password = None
smtp_use_tls = False
subject_prefix = ''
def report(self, exc_data):
msg = self.assemble_email(exc_data)
server = smtplib.SMTP(self.smtp_server)
if self.smtp_use_tls:
server.ehlo()
server.starttls()
server.ehlo()
if self.smtp_username and self.smtp_password:
server.login(self.smtp_username, self.smtp_password)
server.sendmail(self.from_address,
self.to_addresses, msg.as_string())
try:
server.quit()
except sslerror:
# sslerror is raised in tls connections on closing sometimes
pass
def check_params(self):
if not self.to_addresses:
raise ValueError("You must set to_addresses")
if not self.from_address:
raise ValueError("You must set from_address")
if isinstance(self.to_addresses, (str, unicode)):
self.to_addresses = [self.to_addresses]
def assemble_email(self, exc_data):
short_html_version = self.format_html(
exc_data, show_hidden_frames=False)
long_html_version = self.format_html(
exc_data, show_hidden_frames=True)
text_version = self.format_text(
exc_data, show_hidden_frames=False)
msg = MIMEMultipart()
msg.set_type('multipart/alternative')
msg.preamble = msg.epilogue = ''
text_msg = MIMEText(text_version)
text_msg.set_type('text/plain')
text_msg.set_param('charset', 'ASCII')
msg.attach(text_msg)
html_msg = MIMEText(short_html_version)
html_msg.set_type('text/html')
# @@: Correct character set?
html_msg.set_param('charset', 'UTF-8')
html_long = MIMEText(long_html_version)
html_long.set_type('text/html')
html_long.set_param('charset', 'UTF-8')
msg.attach(html_msg)
msg.attach(html_long)
subject = '%s: %s' % (exc_data.exception_type,
formatter.truncate(str(exc_data.exception_value)))
msg['Subject'] = self.subject_prefix + subject
msg['From'] = self.from_address
msg['To'] = ', '.join(self.to_addresses)
return msg
class LogReporter(Reporter):
filename = None
show_hidden_frames = True
def check_params(self):
assert self.filename is not None, (
"You must give a filename")
def report(self, exc_data):
text = self.format_text(
exc_data, show_hidden_frames=self.show_hidden_frames)
f = open(self.filename, 'a')
try:
f.write(text + '\n' + '-'*60 + '\n')
finally:
f.close()
class FileReporter(Reporter):
file = None
show_hidden_frames = True
def check_params(self):
assert self.file is not None, (
"You must give a file object")
def report(self, exc_data):
text = self.format_text(
exc_data, show_hidden_frames=self.show_hidden_frames)
self.file.write(text + '\n' + '-'*60 + '\n')
class WSGIAppReporter(Reporter):
def __init__(self, exc_data):
self.exc_data = exc_data
def __call__(self, environ, start_response):
start_response('500 Server Error', [('Content-type', 'text/html')])
return [formatter.format_html(self.exc_data)]

View File

@@ -0,0 +1,123 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
"""
Creates a human-readable identifier, using numbers and digits,
avoiding ambiguous numbers and letters. hash_identifier can be used
to create compact representations that are unique for a certain string
(or concatenation of strings)
"""
try:
from hashlib import md5
except ImportError:
from md5 import md5
good_characters = "23456789abcdefghjkmnpqrtuvwxyz"
base = len(good_characters)
def make_identifier(number):
"""
Encodes a number as an identifier.
"""
if not isinstance(number, (int, long)):
raise ValueError(
"You can only make identifiers out of integers (not %r)"
% number)
if number < 0:
raise ValueError(
"You cannot make identifiers out of negative numbers: %r"
% number)
result = []
while number:
next = number % base
result.append(good_characters[next])
# Note, this depends on integer rounding of results:
number = number / base
return ''.join(result)
def hash_identifier(s, length, pad=True, hasher=md5, prefix='',
group=None, upper=False):
"""
Hashes the string (with the given hashing module), then turns that
hash into an identifier of the given length (using modulo to
reduce the length of the identifier). If ``pad`` is False, then
the minimum-length identifier will be used; otherwise the
identifier will be padded with 0's as necessary.
``prefix`` will be added last, and does not count towards the
target length. ``group`` will group the characters with ``-`` in
the given lengths, and also does not count towards the target
length. E.g., ``group=4`` will cause a identifier like
``a5f3-hgk3-asdf``. Grouping occurs before the prefix.
"""
if not callable(hasher):
# Accept sha/md5 modules as well as callables
hasher = hasher.new
if length > 26 and hasher is md5:
raise ValueError, (
"md5 cannot create hashes longer than 26 characters in "
"length (you gave %s)" % length)
if isinstance(s, unicode):
s = s.encode('utf-8')
h = hasher(str(s))
bin_hash = h.digest()
modulo = base ** length
number = 0
for c in list(bin_hash):
number = (number * 256 + ord(c)) % modulo
ident = make_identifier(number)
if pad:
ident = good_characters[0]*(length-len(ident)) + ident
if group:
parts = []
while ident:
parts.insert(0, ident[-group:])
ident = ident[:-group]
ident = '-'.join(parts)
if upper:
ident = ident.upper()
return prefix + ident
# doctest tests:
__test__ = {
'make_identifier': """
>>> make_identifier(0)
''
>>> make_identifier(1000)
'c53'
>>> make_identifier(-100)
Traceback (most recent call last):
...
ValueError: You cannot make identifiers out of negative numbers: -100
>>> make_identifier('test')
Traceback (most recent call last):
...
ValueError: You can only make identifiers out of integers (not 'test')
>>> make_identifier(1000000000000)
'c53x9rqh3'
""",
'hash_identifier': """
>>> hash_identifier(0, 5)
'cy2dr'
>>> hash_identifier(0, 10)
'cy2dr6rg46'
>>> hash_identifier('this is a test of a long string', 5)
'awatu'
>>> hash_identifier(0, 26)
'cy2dr6rg46cx8t4w2f3nfexzk4'
>>> hash_identifier(0, 30)
Traceback (most recent call last):
...
ValueError: md5 cannot create hashes longer than 26 characters in length (you gave 30)
>>> hash_identifier(0, 10, group=4)
'cy-2dr6-rg46'
>>> hash_identifier(0, 10, group=4, upper=True, prefix='M-')
'M-CY-2DR6-RG46'
"""}
if __name__ == '__main__':
import doctest
doctest.testmod()

View File

@@ -0,0 +1,354 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
# (c) 2005 Ian Bicking, Clark C. Evans and contributors
# This module is part of the Python Paste Project and is released under
# the MIT License: http://www.opensource.org/licenses/mit-license.php
"""
This module handles sending static content such as in-memory data or
files. At this time it has cache helpers and understands the
if-modified-since request header.
"""
import os, time, mimetypes, zipfile, tarfile
from paste.httpexceptions import *
from paste.httpheaders import *
CACHE_SIZE = 4096
BLOCK_SIZE = 4096 * 16
__all__ = ['DataApp', 'FileApp', 'DirectoryApp', 'ArchiveStore']
class DataApp(object):
"""
Returns an application that will send content in a single chunk,
this application has support for setting cache-control and for
responding to conditional (or HEAD) requests.
Constructor Arguments:
``content`` the content being sent to the client
``headers`` the headers to send /w the response
The remaining ``kwargs`` correspond to headers, where the
underscore is replaced with a dash. These values are only
added to the headers if they are not already provided; thus,
they can be used for default values. Examples include, but
are not limited to:
``content_type``
``content_encoding``
``content_location``
``cache_control()``
This method provides validated construction of the ``Cache-Control``
header as well as providing for automated filling out of the
``EXPIRES`` header for HTTP/1.0 clients.
``set_content()``
This method provides a mechanism to set the content after the
application has been constructed. This method does things
like changing ``Last-Modified`` and ``Content-Length`` headers.
"""
allowed_methods = ('GET', 'HEAD')
def __init__(self, content, headers=None, allowed_methods=None,
**kwargs):
assert isinstance(headers, (type(None), list))
self.expires = None
self.content = None
self.content_length = None
self.last_modified = 0
if allowed_methods is not None:
self.allowed_methods = allowed_methods
self.headers = headers or []
for (k, v) in kwargs.items():
header = get_header(k)
header.update(self.headers, v)
ACCEPT_RANGES.update(self.headers, bytes=True)
if not CONTENT_TYPE(self.headers):
CONTENT_TYPE.update(self.headers)
if content is not None:
self.set_content(content)
def cache_control(self, **kwargs):
self.expires = CACHE_CONTROL.apply(self.headers, **kwargs) or None
return self
def set_content(self, content, last_modified=None):
assert content is not None
if last_modified is None:
self.last_modified = time.time()
else:
self.last_modified = last_modified
self.content = content
self.content_length = len(content)
LAST_MODIFIED.update(self.headers, time=self.last_modified)
return self
def content_disposition(self, **kwargs):
CONTENT_DISPOSITION.apply(self.headers, **kwargs)
return self
def __call__(self, environ, start_response):
method = environ['REQUEST_METHOD'].upper()
if method not in self.allowed_methods:
exc = HTTPMethodNotAllowed(
'You cannot %s a file' % method,
headers=[('Allow', ','.join(self.allowed_methods))])
return exc(environ, start_response)
return self.get(environ, start_response)
def calculate_etag(self):
return '"%s-%s"' % (self.last_modified, self.content_length)
def get(self, environ, start_response):
headers = self.headers[:]
current_etag = self.calculate_etag()
ETAG.update(headers, current_etag)
if self.expires is not None:
EXPIRES.update(headers, delta=self.expires)
try:
client_etags = IF_NONE_MATCH.parse(environ)
if client_etags:
for etag in client_etags:
if etag == current_etag or etag == '*':
# horribly inefficient, n^2 performance, yuck!
for head in list_headers(entity=True):
head.delete(headers)
start_response('304 Not Modified', headers)
return ['']
except HTTPBadRequest, exce:
return exce.wsgi_application(environ, start_response)
# If we get If-None-Match and If-Modified-Since, and
# If-None-Match doesn't match, then we should not try to
# figure out If-Modified-Since (which has 1-second granularity
# and just isn't as accurate)
if not client_etags:
try:
client_clock = IF_MODIFIED_SINCE.parse(environ)
if client_clock >= int(self.last_modified):
# horribly inefficient, n^2 performance, yuck!
for head in list_headers(entity=True):
head.delete(headers)
start_response('304 Not Modified', headers)
return [''] # empty body
except HTTPBadRequest, exce:
return exce.wsgi_application(environ, start_response)
(lower, upper) = (0, self.content_length - 1)
range = RANGE.parse(environ)
if range and 'bytes' == range[0] and 1 == len(range[1]):
(lower, upper) = range[1][0]
upper = upper or (self.content_length - 1)
if upper >= self.content_length or lower > upper:
return HTTPRequestRangeNotSatisfiable((
"Range request was made beyond the end of the content,\r\n"
"which is %s long.\r\n Range: %s\r\n") % (
self.content_length, RANGE(environ))
).wsgi_application(environ, start_response)
content_length = upper - lower + 1
CONTENT_RANGE.update(headers, first_byte=lower, last_byte=upper,
total_length = self.content_length)
CONTENT_LENGTH.update(headers, content_length)
if content_length == self.content_length:
start_response('200 OK', headers)
else:
start_response('206 Partial Content', headers)
if self.content is not None:
return [self.content[lower:upper+1]]
return (lower, content_length)
class FileApp(DataApp):
"""
Returns an application that will send the file at the given
filename. Adds a mime type based on ``mimetypes.guess_type()``.
See DataApp for the arguments beyond ``filename``.
"""
def __init__(self, filename, headers=None, **kwargs):
self.filename = filename
content_type, content_encoding = self.guess_type()
if content_type and 'content_type' not in kwargs:
kwargs['content_type'] = content_type
if content_encoding and 'content_encoding' not in kwargs:
kwargs['content_encoding'] = content_encoding
DataApp.__init__(self, None, headers, **kwargs)
def guess_type(self):
return mimetypes.guess_type(self.filename)
def update(self, force=False):
stat = os.stat(self.filename)
if not force and stat.st_mtime == self.last_modified:
return
self.last_modified = stat.st_mtime
if stat.st_size < CACHE_SIZE:
fh = open(self.filename,"rb")
self.set_content(fh.read(), stat.st_mtime)
fh.close()
else:
self.content = None
self.content_length = stat.st_size
# This is updated automatically if self.set_content() is
# called
LAST_MODIFIED.update(self.headers, time=self.last_modified)
def get(self, environ, start_response):
is_head = environ['REQUEST_METHOD'].upper() == 'HEAD'
if 'max-age=0' in CACHE_CONTROL(environ).lower():
self.update(force=True) # RFC 2616 13.2.6
else:
self.update()
if not self.content:
if not os.path.exists(self.filename):
exc = HTTPNotFound(
'The resource does not exist',
comment="No file at %r" % self.filename)
return exc(environ, start_response)
try:
file = open(self.filename, 'rb')
except (IOError, OSError), e:
exc = HTTPForbidden(
'You are not permitted to view this file (%s)' % e)
return exc.wsgi_application(
environ, start_response)
retval = DataApp.get(self, environ, start_response)
if isinstance(retval, list):
# cached content, exception, or not-modified
if is_head:
return ['']
return retval
(lower, content_length) = retval
if is_head:
return ['']
file.seek(lower)
file_wrapper = environ.get('wsgi.file_wrapper', None)
if file_wrapper:
return file_wrapper(file, BLOCK_SIZE)
else:
return _FileIter(file, size=content_length)
class _FileIter(object):
def __init__(self, file, block_size=None, size=None):
self.file = file
self.size = size
self.block_size = block_size or BLOCK_SIZE
def __iter__(self):
return self
def next(self):
chunk_size = self.block_size
if self.size is not None:
if chunk_size > self.size:
chunk_size = self.size
self.size -= chunk_size
data = self.file.read(chunk_size)
if not data:
raise StopIteration
return data
def close(self):
self.file.close()
class DirectoryApp(object):
"""
Returns an application that dispatches requests to corresponding FileApps based on PATH_INFO.
FileApp instances are cached. This app makes sure not to serve any files that are not in a subdirectory.
To customize FileApp creation override ``DirectoryApp.make_fileapp``
"""
def __init__(self, path):
self.path = os.path.abspath(path)
if not self.path.endswith(os.path.sep):
self.path += os.path.sep
assert os.path.isdir(self.path)
self.cached_apps = {}
make_fileapp = FileApp
def __call__(self, environ, start_response):
path_info = environ['PATH_INFO']
app = self.cached_apps.get(path_info)
if app is None:
path = os.path.join(self.path, path_info.lstrip('/'))
if not os.path.normpath(path).startswith(self.path):
app = HTTPForbidden()
elif os.path.isfile(path):
app = self.make_fileapp(path)
self.cached_apps[path_info] = app
else:
app = HTTPNotFound(comment=path)
return app(environ, start_response)
class ArchiveStore(object):
"""
Returns an application that serves up a DataApp for items requested
in a given zip or tar archive.
Constructor Arguments:
``filepath`` the path to the archive being served
``cache_control()``
This method provides validated construction of the ``Cache-Control``
header as well as providing for automated filling out of the
``EXPIRES`` header for HTTP/1.0 clients.
"""
def __init__(self, filepath):
if zipfile.is_zipfile(filepath):
self.archive = zipfile.ZipFile(filepath,"r")
elif tarfile.is_tarfile(filepath):
self.archive = tarfile.TarFileCompat(filepath,"r")
else:
raise AssertionError("filepath '%s' is not a zip or tar " % filepath)
self.expires = None
self.last_modified = time.time()
self.cache = {}
def cache_control(self, **kwargs):
self.expires = CACHE_CONTROL.apply(self.headers, **kwargs) or None
return self
def __call__(self, environ, start_response):
path = environ.get("PATH_INFO","")
if path.startswith("/"):
path = path[1:]
application = self.cache.get(path)
if application:
return application(environ, start_response)
try:
info = self.archive.getinfo(path)
except KeyError:
exc = HTTPNotFound("The file requested, '%s', was not found." % path)
return exc.wsgi_application(environ, start_response)
if info.filename.endswith("/"):
exc = HTTPNotFound("Path requested, '%s', is not a file." % path)
return exc.wsgi_application(environ, start_response)
content_type, content_encoding = mimetypes.guess_type(info.filename)
# 'None' is not a valid content-encoding, so don't set the header if
# mimetypes.guess_type returns None
if content_encoding is not None:
app = DataApp(None, content_type = content_type,
content_encoding = content_encoding)
else:
app = DataApp(None, content_type = content_type)
app.set_content(self.archive.read(path),
time.mktime(info.date_time + (0,0,0)))
self.cache[path] = app
app.expires = self.expires
return app(environ, start_response)

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,108 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
"""
Creates a session object.
In your application, use::
environ['paste.flup_session_service'].session
This will return a dictionary. The contents of this dictionary will
be saved to disk when the request is completed. The session will be
created when you first fetch the session dictionary, and a cookie will
be sent in that case. There's current no way to use sessions without
cookies, and there's no way to delete a session except to clear its
data.
"""
from paste import httpexceptions
from paste import wsgilib
import flup.middleware.session
flup_session = flup.middleware.session
# This is a dictionary of existing stores, keyed by a tuple of
# store type and parameters
store_cache = {}
class NoDefault(object):
pass
class SessionMiddleware(object):
session_classes = {
'memory': (flup_session.MemorySessionStore,
[('session_timeout', 'timeout', int, 60)]),
'disk': (flup_session.DiskSessionStore,
[('session_timeout', 'timeout', int, 60),
('session_dir', 'storeDir', str, '/tmp/sessions')]),
'shelve': (flup_session.ShelveSessionStore,
[('session_timeout', 'timeout', int, 60),
('session_file', 'storeFile', str,
'/tmp/session.shelve')]),
}
def __init__(self, app,
global_conf=None,
session_type=NoDefault,
cookie_name=NoDefault,
**store_config
):
self.application = app
if session_type is NoDefault:
session_type = global_conf.get('session_type', 'disk')
self.session_type = session_type
try:
self.store_class, self.store_args = self.session_classes[self.session_type]
except KeyError:
raise KeyError(
"The session_type %s is unknown (I know about %s)"
% (self.session_type,
', '.join(self.session_classes.keys())))
kw = {}
for config_name, kw_name, coercer, default in self.store_args:
value = coercer(store_config.get(config_name, default))
kw[kw_name] = value
self.store = self.store_class(**kw)
if cookie_name is NoDefault:
cookie_name = global_conf.get('session_cookie', '_SID_')
self.cookie_name = cookie_name
def __call__(self, environ, start_response):
service = flup_session.SessionService(
self.store, environ, cookieName=self.cookie_name,
fieldName=self.cookie_name)
environ['paste.flup_session_service'] = service
def cookie_start_response(status, headers, exc_info=None):
service.addCookie(headers)
return start_response(status, headers, exc_info)
try:
app_iter = self.application(environ, cookie_start_response)
except httpexceptions.HTTPException, e:
headers = (e.headers or {}).items()
service.addCookie(headers)
e.headers = dict(headers)
service.close()
raise
except:
service.close()
raise
return wsgilib.add_close(app_iter, service.close)
def make_session_middleware(app, global_conf,
session_type=NoDefault,
cookie_name=NoDefault,
**store_config):
"""
Wraps the application in a session-managing middleware.
The session service can then be found in
``environ['paste.flup_session_service']``
"""
return SessionMiddleware(
app, global_conf=global_conf,
session_type=session_type, cookie_name=cookie_name,
**store_config)

View File

@@ -0,0 +1,111 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
"""
WSGI middleware
Gzip-encodes the response.
"""
import gzip
from paste.response import header_value, remove_header
from paste.httpheaders import CONTENT_LENGTH
try:
from cStringIO import StringIO
except ImportError:
from StringIO import StringIO
class GzipOutput(object):
pass
class middleware(object):
def __init__(self, application, compress_level=6):
self.application = application
self.compress_level = int(compress_level)
def __call__(self, environ, start_response):
if 'gzip' not in environ.get('HTTP_ACCEPT_ENCODING', ''):
# nothing for us to do, so this middleware will
# be a no-op:
return self.application(environ, start_response)
response = GzipResponse(start_response, self.compress_level)
app_iter = self.application(environ,
response.gzip_start_response)
if app_iter is not None:
response.finish_response(app_iter)
return response.write()
class GzipResponse(object):
def __init__(self, start_response, compress_level):
self.start_response = start_response
self.compress_level = compress_level
self.buffer = StringIO()
self.compressible = False
self.content_length = None
def gzip_start_response(self, status, headers, exc_info=None):
self.headers = headers
ct = header_value(headers,'content-type')
ce = header_value(headers,'content-encoding')
self.compressible = False
if ct and (ct.startswith('text/') or ct.startswith('application/')) \
and 'zip' not in ct:
self.compressible = True
if ce:
self.compressible = False
if self.compressible:
headers.append(('content-encoding', 'gzip'))
remove_header(headers, 'content-length')
self.headers = headers
self.status = status
return self.buffer.write
def write(self):
out = self.buffer
out.seek(0)
s = out.getvalue()
out.close()
return [s]
def finish_response(self, app_iter):
if self.compressible:
output = gzip.GzipFile(mode='wb', compresslevel=self.compress_level,
fileobj=self.buffer)
else:
output = self.buffer
try:
for s in app_iter:
output.write(s)
if self.compressible:
output.close()
finally:
if hasattr(app_iter, 'close'):
app_iter.close()
content_length = self.buffer.tell()
CONTENT_LENGTH.update(self.headers, content_length)
self.start_response(self.status, self.headers)
def filter_factory(application, **conf):
import warnings
warnings.warn(
'This function is deprecated; use make_gzip_middleware instead',
DeprecationWarning, 2)
def filter(application):
return middleware(application)
return filter
def make_gzip_middleware(app, global_conf, compress_level=6):
"""
Wrap the middleware, so that it applies gzipping to a response
when it is supported by the browser and the content is of
type ``text/*`` or ``application/*``
"""
compress_level = int(compress_level)
return middleware(app, compress_level=compress_level)

View File

@@ -0,0 +1,660 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
# (c) 2005 Ian Bicking, Clark C. Evans and contributors
# This module is part of the Python Paste Project and is released under
# the MIT License: http://www.opensource.org/licenses/mit-license.php
# Some of this code was funded by http://prometheusresearch.com
"""
HTTP Exception Middleware
This module processes Python exceptions that relate to HTTP exceptions
by defining a set of exceptions, all subclasses of HTTPException, and a
request handler (`middleware`) that catches these exceptions and turns
them into proper responses.
This module defines exceptions according to RFC 2068 [1]_ : codes with
100-300 are not really errors; 400's are client errors, and 500's are
server errors. According to the WSGI specification [2]_ , the application
can call ``start_response`` more then once only under two conditions:
(a) the response has not yet been sent, or (b) if the second and
subsequent invocations of ``start_response`` have a valid ``exc_info``
argument obtained from ``sys.exc_info()``. The WSGI specification then
requires the server or gateway to handle the case where content has been
sent and then an exception was encountered.
Exceptions in the 5xx range and those raised after ``start_response``
has been called are treated as serious errors and the ``exc_info`` is
filled-in with information needed for a lower level module to generate a
stack trace and log information.
Exception
HTTPException
HTTPRedirection
* 300 - HTTPMultipleChoices
* 301 - HTTPMovedPermanently
* 302 - HTTPFound
* 303 - HTTPSeeOther
* 304 - HTTPNotModified
* 305 - HTTPUseProxy
* 306 - Unused (not implemented, obviously)
* 307 - HTTPTemporaryRedirect
HTTPError
HTTPClientError
* 400 - HTTPBadRequest
* 401 - HTTPUnauthorized
* 402 - HTTPPaymentRequired
* 403 - HTTPForbidden
* 404 - HTTPNotFound
* 405 - HTTPMethodNotAllowed
* 406 - HTTPNotAcceptable
* 407 - HTTPProxyAuthenticationRequired
* 408 - HTTPRequestTimeout
* 409 - HTTPConfict
* 410 - HTTPGone
* 411 - HTTPLengthRequired
* 412 - HTTPPreconditionFailed
* 413 - HTTPRequestEntityTooLarge
* 414 - HTTPRequestURITooLong
* 415 - HTTPUnsupportedMediaType
* 416 - HTTPRequestRangeNotSatisfiable
* 417 - HTTPExpectationFailed
HTTPServerError
* 500 - HTTPInternalServerError
* 501 - HTTPNotImplemented
* 502 - HTTPBadGateway
* 503 - HTTPServiceUnavailable
* 504 - HTTPGatewayTimeout
* 505 - HTTPVersionNotSupported
References:
.. [1] http://www.python.org/peps/pep-0333.html#error-handling
.. [2] http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.5
"""
import types
from paste.wsgilib import catch_errors_app
from paste.response import has_header, header_value, replace_header
from paste.request import resolve_relative_url
from paste.util.quoting import strip_html, html_quote, no_quote, comment_quote
SERVER_NAME = 'WSGI Server'
TEMPLATE = """\
<html>\r
<head><title>%(title)s</title></head>\r
<body>\r
<h1>%(title)s</h1>\r
<p>%(body)s</p>\r
<hr noshade>\r
<div align="right">%(server)s</div>\r
</body>\r
</html>\r
"""
class HTTPException(Exception):
"""
the HTTP exception base class
This encapsulates an HTTP response that interrupts normal application
flow; but one which is not necessarly an error condition. For
example, codes in the 300's are exceptions in that they interrupt
normal processing; however, they are not considered errors.
This class is complicated by 4 factors:
1. The content given to the exception may either be plain-text or
as html-text.
2. The template may want to have string-substitutions taken from
the current ``environ`` or values from incoming headers. This
is especially troublesome due to case sensitivity.
3. The final output may either be text/plain or text/html
mime-type as requested by the client application.
4. Each exception has a default explanation, but those who
raise exceptions may want to provide additional detail.
Attributes:
``code``
the HTTP status code for the exception
``title``
remainder of the status line (stuff after the code)
``explanation``
a plain-text explanation of the error message that is
not subject to environment or header substitutions;
it is accessible in the template via %(explanation)s
``detail``
a plain-text message customization that is not subject
to environment or header substitutions; accessible in
the template via %(detail)s
``template``
a content fragment (in HTML) used for environment and
header substitution; the default template includes both
the explanation and further detail provided in the
message
``required_headers``
a sequence of headers which are required for proper
construction of the exception
Parameters:
``detail``
a plain-text override of the default ``detail``
``headers``
a list of (k,v) header pairs
``comment``
a plain-text additional information which is
usually stripped/hidden for end-users
To override the template (which is HTML content) or the plain-text
explanation, one must subclass the given exception; or customize it
after it has been created. This particular breakdown of a message
into explanation, detail and template allows both the creation of
plain-text and html messages for various clients as well as
error-free substitution of environment variables and headers.
"""
code = None
title = None
explanation = ''
detail = ''
comment = ''
template = "%(explanation)s\r\n<br/>%(detail)s\r\n<!-- %(comment)s -->"
required_headers = ()
def __init__(self, detail=None, headers=None, comment=None):
assert self.code, "Do not directly instantiate abstract exceptions."
assert isinstance(headers, (type(None), list)), (
"headers must be None or a list: %r"
% headers)
assert isinstance(detail, (type(None), basestring)), (
"detail must be None or a string: %r" % detail)
assert isinstance(comment, (type(None), basestring)), (
"comment must be None or a string: %r" % comment)
self.headers = headers or tuple()
for req in self.required_headers:
assert headers and has_header(headers, req), (
"Exception %s must be passed the header %r "
"(got headers: %r)"
% (self.__class__.__name__, req, headers))
if detail is not None:
self.detail = detail
if comment is not None:
self.comment = comment
Exception.__init__(self,"%s %s\n%s\n%s\n" % (
self.code, self.title, self.explanation, self.detail))
def make_body(self, environ, template, escfunc, comment_escfunc=None):
comment_escfunc = comment_escfunc or escfunc
args = {'explanation': escfunc(self.explanation),
'detail': escfunc(self.detail),
'comment': comment_escfunc(self.comment)}
if HTTPException.template != self.template:
for (k, v) in environ.items():
args[k] = escfunc(v)
if self.headers:
for (k, v) in self.headers:
args[k.lower()] = escfunc(v)
for key, value in args.items():
if isinstance(value, unicode):
args[key] = value.encode('utf8', 'xmlcharrefreplace')
return template % args
def plain(self, environ):
""" text/plain representation of the exception """
body = self.make_body(environ, strip_html(self.template), no_quote, comment_quote)
return ('%s %s\r\n%s\r\n' % (self.code, self.title, body))
def html(self, environ):
""" text/html representation of the exception """
body = self.make_body(environ, self.template, html_quote, comment_quote)
return TEMPLATE % {
'title': self.title,
'code': self.code,
'server': SERVER_NAME,
'body': body }
def prepare_content(self, environ):
if self.headers:
headers = list(self.headers)
else:
headers = []
if 'html' in environ.get('HTTP_ACCEPT','') or \
'*/*' in environ.get('HTTP_ACCEPT',''):
replace_header(headers, 'content-type', 'text/html')
content = self.html(environ)
else:
replace_header(headers, 'content-type', 'text/plain')
content = self.plain(environ)
if isinstance(content, unicode):
content = content.encode('utf8')
cur_content_type = (
header_value(headers, 'content-type')
or 'text/html')
replace_header(
headers, 'content-type',
cur_content_type + '; charset=utf8')
return headers, content
def response(self, environ):
from paste.wsgiwrappers import WSGIResponse
headers, content = self.prepare_content(environ)
resp = WSGIResponse(code=self.code, content=content)
resp.headers = resp.headers.fromlist(headers)
return resp
def wsgi_application(self, environ, start_response, exc_info=None):
"""
This exception as a WSGI application
"""
headers, content = self.prepare_content(environ)
start_response('%s %s' % (self.code, self.title),
headers,
exc_info)
return [content]
__call__ = wsgi_application
def __repr__(self):
return '<%s %s; code=%s>' % (self.__class__.__name__,
self.title, self.code)
class HTTPError(HTTPException):
"""
base class for status codes in the 400's and 500's
This is an exception which indicates that an error has occurred,
and that any work in progress should not be committed. These are
typically results in the 400's and 500's.
"""
#
# 3xx Redirection
#
# This class of status code indicates that further action needs to be
# taken by the user agent in order to fulfill the request. The action
# required MAY be carried out by the user agent without interaction with
# the user if and only if the method used in the second request is GET or
# HEAD. A client SHOULD detect infinite redirection loops, since such
# loops generate network traffic for each redirection.
#
class HTTPRedirection(HTTPException):
"""
base class for 300's status code (redirections)
This is an abstract base class for 3xx redirection. It indicates
that further action needs to be taken by the user agent in order
to fulfill the request. It does not necessarly signal an error
condition.
"""
class _HTTPMove(HTTPRedirection):
"""
redirections which require a Location field
Since a 'Location' header is a required attribute of 301, 302, 303,
305 and 307 (but not 304), this base class provides the mechanics to
make this easy. While this has the same parameters as HTTPException,
if a location is not provided in the headers; it is assumed that the
detail _is_ the location (this for backward compatibility, otherwise
we'd add a new attribute).
"""
required_headers = ('location',)
explanation = 'The resource has been moved to'
template = (
'%(explanation)s <a href="%(location)s">%(location)s</a>;\r\n'
'you should be redirected automatically.\r\n'
'%(detail)s\r\n<!-- %(comment)s -->')
def __init__(self, detail=None, headers=None, comment=None):
assert isinstance(headers, (type(None), list))
headers = headers or []
location = header_value(headers,'location')
if not location:
location = detail
detail = ''
headers.append(('location', location))
assert location, ("HTTPRedirection specified neither a "
"location in the headers nor did it "
"provide a detail argument.")
HTTPRedirection.__init__(self, location, headers, comment)
if detail is not None:
self.detail = detail
def relative_redirect(cls, dest_uri, environ, detail=None, headers=None, comment=None):
"""
Create a redirect object with the dest_uri, which may be relative,
considering it relative to the uri implied by the given environ.
"""
location = resolve_relative_url(dest_uri, environ)
headers = headers or []
headers.append(('Location', location))
return cls(detail=detail, headers=headers, comment=comment)
relative_redirect = classmethod(relative_redirect)
def location(self):
for name, value in self.headers:
if name.lower() == 'location':
return value
else:
raise KeyError("No location set for %s" % self)
class HTTPMultipleChoices(_HTTPMove):
code = 300
title = 'Multiple Choices'
class HTTPMovedPermanently(_HTTPMove):
code = 301
title = 'Moved Permanently'
class HTTPFound(_HTTPMove):
code = 302
title = 'Found'
explanation = 'The resource was found at'
# This one is safe after a POST (the redirected location will be
# retrieved with GET):
class HTTPSeeOther(_HTTPMove):
code = 303
title = 'See Other'
class HTTPNotModified(HTTPRedirection):
# @@: but not always (HTTP section 14.18.1)...?
# @@: Removed 'date' requirement, as its not required for an ETag
# @@: FIXME: This should require either an ETag or a date header
code = 304
title = 'Not Modified'
message = ''
# @@: should include date header, optionally other headers
# @@: should not return a content body
def plain(self, environ):
return ''
def html(self, environ):
""" text/html representation of the exception """
return ''
class HTTPUseProxy(_HTTPMove):
# @@: OK, not a move, but looks a little like one
code = 305
title = 'Use Proxy'
explanation = (
'The resource must be accessed through a proxy '
'located at')
class HTTPTemporaryRedirect(_HTTPMove):
code = 307
title = 'Temporary Redirect'
#
# 4xx Client Error
#
# The 4xx class of status code is intended for cases in which the client
# seems to have erred. Except when responding to a HEAD request, the
# server SHOULD include an entity containing an explanation of the error
# situation, and whether it is a temporary or permanent condition. These
# status codes are applicable to any request method. User agents SHOULD
# display any included entity to the user.
#
class HTTPClientError(HTTPError):
"""
base class for the 400's, where the client is in-error
This is an error condition in which the client is presumed to be
in-error. This is an expected problem, and thus is not considered
a bug. A server-side traceback is not warranted. Unless specialized,
this is a '400 Bad Request'
"""
code = 400
title = 'Bad Request'
explanation = ('The server could not comply with the request since\r\n'
'it is either malformed or otherwise incorrect.\r\n')
class HTTPBadRequest(HTTPClientError):
pass
class HTTPUnauthorized(HTTPClientError):
code = 401
title = 'Unauthorized'
explanation = (
'This server could not verify that you are authorized to\r\n'
'access the document you requested. Either you supplied the\r\n'
'wrong credentials (e.g., bad password), or your browser\r\n'
'does not understand how to supply the credentials required.\r\n')
class HTTPPaymentRequired(HTTPClientError):
code = 402
title = 'Payment Required'
explanation = ('Access was denied for financial reasons.')
class HTTPForbidden(HTTPClientError):
code = 403
title = 'Forbidden'
explanation = ('Access was denied to this resource.')
class HTTPNotFound(HTTPClientError):
code = 404
title = 'Not Found'
explanation = ('The resource could not be found.')
class HTTPMethodNotAllowed(HTTPClientError):
required_headers = ('allow',)
code = 405
title = 'Method Not Allowed'
# override template since we need an environment variable
template = ('The method %(REQUEST_METHOD)s is not allowed for '
'this resource.\r\n%(detail)s')
class HTTPNotAcceptable(HTTPClientError):
code = 406
title = 'Not Acceptable'
# override template since we need an environment variable
template = ('The resource could not be generated that was '
'acceptable to your browser (content\r\nof type '
'%(HTTP_ACCEPT)s).\r\n%(detail)s')
class HTTPProxyAuthenticationRequired(HTTPClientError):
code = 407
title = 'Proxy Authentication Required'
explanation = ('Authentication /w a local proxy is needed.')
class HTTPRequestTimeout(HTTPClientError):
code = 408
title = 'Request Timeout'
explanation = ('The server has waited too long for the request to '
'be sent by the client.')
class HTTPConflict(HTTPClientError):
code = 409
title = 'Conflict'
explanation = ('There was a conflict when trying to complete '
'your request.')
class HTTPGone(HTTPClientError):
code = 410
title = 'Gone'
explanation = ('This resource is no longer available. No forwarding '
'address is given.')
class HTTPLengthRequired(HTTPClientError):
code = 411
title = 'Length Required'
explanation = ('Content-Length header required.')
class HTTPPreconditionFailed(HTTPClientError):
code = 412
title = 'Precondition Failed'
explanation = ('Request precondition failed.')
class HTTPRequestEntityTooLarge(HTTPClientError):
code = 413
title = 'Request Entity Too Large'
explanation = ('The body of your request was too large for this server.')
class HTTPRequestURITooLong(HTTPClientError):
code = 414
title = 'Request-URI Too Long'
explanation = ('The request URI was too long for this server.')
class HTTPUnsupportedMediaType(HTTPClientError):
code = 415
title = 'Unsupported Media Type'
# override template since we need an environment variable
template = ('The request media type %(CONTENT_TYPE)s is not '
'supported by this server.\r\n%(detail)s')
class HTTPRequestRangeNotSatisfiable(HTTPClientError):
code = 416
title = 'Request Range Not Satisfiable'
explanation = ('The Range requested is not available.')
class HTTPExpectationFailed(HTTPClientError):
code = 417
title = 'Expectation Failed'
explanation = ('Expectation failed.')
#
# 5xx Server Error
#
# Response status codes beginning with the digit "5" indicate cases in
# which the server is aware that it has erred or is incapable of
# performing the request. Except when responding to a HEAD request, the
# server SHOULD include an entity containing an explanation of the error
# situation, and whether it is a temporary or permanent condition. User
# agents SHOULD display any included entity to the user. These response
# codes are applicable to any request method.
#
class HTTPServerError(HTTPError):
"""
base class for the 500's, where the server is in-error
This is an error condition in which the server is presumed to be
in-error. This is usually unexpected, and thus requires a traceback;
ideally, opening a support ticket for the customer. Unless specialized,
this is a '500 Internal Server Error'
"""
code = 500
title = 'Internal Server Error'
explanation = (
'The server has either erred or is incapable of performing\r\n'
'the requested operation.\r\n')
class HTTPInternalServerError(HTTPServerError):
pass
class HTTPNotImplemented(HTTPServerError):
code = 501
title = 'Not Implemented'
# override template since we need an environment variable
template = ('The request method %(REQUEST_METHOD)s is not implemented '
'for this server.\r\n%(detail)s')
class HTTPBadGateway(HTTPServerError):
code = 502
title = 'Bad Gateway'
explanation = ('Bad gateway.')
class HTTPServiceUnavailable(HTTPServerError):
code = 503
title = 'Service Unavailable'
explanation = ('The server is currently unavailable. '
'Please try again at a later time.')
class HTTPGatewayTimeout(HTTPServerError):
code = 504
title = 'Gateway Timeout'
explanation = ('The gateway has timed out.')
class HTTPVersionNotSupported(HTTPServerError):
code = 505
title = 'HTTP Version Not Supported'
explanation = ('The HTTP version is not supported.')
# abstract HTTP related exceptions
__all__ = ['HTTPException', 'HTTPRedirection', 'HTTPError' ]
_exceptions = {}
for name, value in globals().items():
if (isinstance(value, (type, types.ClassType)) and
issubclass(value, HTTPException) and
value.code):
_exceptions[value.code] = value
__all__.append(name)
def get_exception(code):
return _exceptions[code]
############################################################
## Middleware implementation:
############################################################
class HTTPExceptionHandler(object):
"""
catches exceptions and turns them into proper HTTP responses
This middleware catches any exceptions (which are subclasses of
``HTTPException``) and turns them into proper HTTP responses.
Note if the headers have already been sent, the stack trace is
always maintained as this indicates a programming error.
Note that you must raise the exception before returning the
app_iter, and you cannot use this with generator apps that don't
raise an exception until after their app_iter is iterated over.
"""
def __init__(self, application, warning_level=None):
assert not warning_level or ( warning_level > 99 and
warning_level < 600)
if warning_level is not None:
import warnings
warnings.warn('The warning_level parameter is not used or supported',
DeprecationWarning, 2)
self.warning_level = warning_level or 500
self.application = application
def __call__(self, environ, start_response):
environ['paste.httpexceptions'] = self
environ.setdefault('paste.expected_exceptions',
[]).append(HTTPException)
try:
return self.application(environ, start_response)
except HTTPException, exc:
return exc(environ, start_response)
def middleware(*args, **kw):
import warnings
# deprecated 13 dec 2005
warnings.warn('httpexceptions.middleware is deprecated; use '
'make_middleware or HTTPExceptionHandler instead',
DeprecationWarning, 2)
return make_middleware(*args, **kw)
def make_middleware(app, global_conf=None, warning_level=None):
"""
``httpexceptions`` middleware; this catches any
``paste.httpexceptions.HTTPException`` exceptions (exceptions like
``HTTPNotFound``, ``HTTPMovedPermanently``, etc) and turns them
into proper HTTP responses.
``warning_level`` can be an integer corresponding to an HTTP code.
Any code over that value will be passed 'up' the chain, potentially
reported on by another piece of middleware.
"""
if warning_level:
warning_level = int(warning_level)
return HTTPExceptionHandler(app, warning_level=warning_level)
__all__.extend(['HTTPExceptionHandler', 'get_exception'])

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,436 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
# Also licenced under the Apache License, 2.0: http://opensource.org/licenses/apache2.0.php
# Licensed to PSF under a Contributor Agreement
"""
Middleware to check for obedience to the WSGI specification.
Some of the things this checks:
* Signature of the application and start_response (including that
keyword arguments are not used).
* Environment checks:
- Environment is a dictionary (and not a subclass).
- That all the required keys are in the environment: REQUEST_METHOD,
SERVER_NAME, SERVER_PORT, wsgi.version, wsgi.input, wsgi.errors,
wsgi.multithread, wsgi.multiprocess, wsgi.run_once
- That HTTP_CONTENT_TYPE and HTTP_CONTENT_LENGTH are not in the
environment (these headers should appear as CONTENT_LENGTH and
CONTENT_TYPE).
- Warns if QUERY_STRING is missing, as the cgi module acts
unpredictably in that case.
- That CGI-style variables (that don't contain a .) have
(non-unicode) string values
- That wsgi.version is a tuple
- That wsgi.url_scheme is 'http' or 'https' (@@: is this too
restrictive?)
- Warns if the REQUEST_METHOD is not known (@@: probably too
restrictive).
- That SCRIPT_NAME and PATH_INFO are empty or start with /
- That at least one of SCRIPT_NAME or PATH_INFO are set.
- That CONTENT_LENGTH is a positive integer.
- That SCRIPT_NAME is not '/' (it should be '', and PATH_INFO should
be '/').
- That wsgi.input has the methods read, readline, readlines, and
__iter__
- That wsgi.errors has the methods flush, write, writelines
* The status is a string, contains a space, starts with an integer,
and that integer is in range (> 100).
* That the headers is a list (not a subclass, not another kind of
sequence).
* That the items of the headers are tuples of strings.
* That there is no 'status' header (that is used in CGI, but not in
WSGI).
* That the headers don't contain newlines or colons, end in _ or -, or
contain characters codes below 037.
* That Content-Type is given if there is content (CGI often has a
default content type, but WSGI does not).
* That no Content-Type is given when there is no content (@@: is this
too restrictive?)
* That the exc_info argument to start_response is a tuple or None.
* That all calls to the writer are with strings, and no other methods
on the writer are accessed.
* That wsgi.input is used properly:
- .read() is called with zero or one argument
- That it returns a string
- That readline, readlines, and __iter__ return strings
- That .close() is not called
- No other methods are provided
* That wsgi.errors is used properly:
- .write() and .writelines() is called with a string
- That .close() is not called, and no other methods are provided.
* The response iterator:
- That it is not a string (it should be a list of a single string; a
string will work, but perform horribly).
- That .next() returns a string
- That the iterator is not iterated over until start_response has
been called (that can signal either a server or application
error).
- That .close() is called (doesn't raise exception, only prints to
sys.stderr, because we only know it isn't called when the object
is garbage collected).
"""
import re
import sys
from types import DictType, StringType, TupleType, ListType
import warnings
header_re = re.compile(r'^[a-zA-Z][a-zA-Z0-9\-_]*$')
bad_header_value_re = re.compile(r'[\000-\037]')
class WSGIWarning(Warning):
"""
Raised in response to WSGI-spec-related warnings
"""
def middleware(application, global_conf=None):
"""
When applied between a WSGI server and a WSGI application, this
middleware will check for WSGI compliancy on a number of levels.
This middleware does not modify the request or response in any
way, but will throw an AssertionError if anything seems off
(except for a failure to close the application iterator, which
will be printed to stderr -- there's no way to throw an exception
at that point).
"""
def lint_app(*args, **kw):
assert len(args) == 2, "Two arguments required"
assert not kw, "No keyword arguments allowed"
environ, start_response = args
check_environ(environ)
# We use this to check if the application returns without
# calling start_response:
start_response_started = []
def start_response_wrapper(*args, **kw):
assert len(args) == 2 or len(args) == 3, (
"Invalid number of arguments: %s" % args)
assert not kw, "No keyword arguments allowed"
status = args[0]
headers = args[1]
if len(args) == 3:
exc_info = args[2]
else:
exc_info = None
check_status(status)
check_headers(headers)
check_content_type(status, headers)
check_exc_info(exc_info)
start_response_started.append(None)
return WriteWrapper(start_response(*args))
environ['wsgi.input'] = InputWrapper(environ['wsgi.input'])
environ['wsgi.errors'] = ErrorWrapper(environ['wsgi.errors'])
iterator = application(environ, start_response_wrapper)
assert iterator is not None and iterator != False, (
"The application must return an iterator, if only an empty list")
check_iterator(iterator)
return IteratorWrapper(iterator, start_response_started)
return lint_app
class InputWrapper(object):
def __init__(self, wsgi_input):
self.input = wsgi_input
def read(self, *args):
assert len(args) <= 1
v = self.input.read(*args)
assert type(v) is type("")
return v
def readline(self, *args):
v = self.input.readline(*args)
assert type(v) is type("")
return v
def readlines(self, *args):
assert len(args) <= 1
lines = self.input.readlines(*args)
assert type(lines) is type([])
for line in lines:
assert type(line) is type("")
return lines
def __iter__(self):
while 1:
line = self.readline()
if not line:
return
yield line
def close(self):
assert 0, "input.close() must not be called"
class ErrorWrapper(object):
def __init__(self, wsgi_errors):
self.errors = wsgi_errors
def write(self, s):
assert type(s) is type("")
self.errors.write(s)
def flush(self):
self.errors.flush()
def writelines(self, seq):
for line in seq:
self.write(line)
def close(self):
assert 0, "errors.close() must not be called"
class WriteWrapper(object):
def __init__(self, wsgi_writer):
self.writer = wsgi_writer
def __call__(self, s):
assert type(s) is type("")
self.writer(s)
class PartialIteratorWrapper(object):
def __init__(self, wsgi_iterator):
self.iterator = wsgi_iterator
def __iter__(self):
# We want to make sure __iter__ is called
return IteratorWrapper(self.iterator)
class IteratorWrapper(object):
def __init__(self, wsgi_iterator, check_start_response):
self.original_iterator = wsgi_iterator
self.iterator = iter(wsgi_iterator)
self.closed = False
self.check_start_response = check_start_response
def __iter__(self):
return self
def next(self):
assert not self.closed, (
"Iterator read after closed")
v = self.iterator.next()
if self.check_start_response is not None:
assert self.check_start_response, (
"The application returns and we started iterating over its body, but start_response has not yet been called")
self.check_start_response = None
return v
def close(self):
self.closed = True
if hasattr(self.original_iterator, 'close'):
self.original_iterator.close()
def __del__(self):
if not self.closed:
sys.stderr.write(
"Iterator garbage collected without being closed")
assert self.closed, (
"Iterator garbage collected without being closed")
def check_environ(environ):
assert type(environ) is DictType, (
"Environment is not of the right type: %r (environment: %r)"
% (type(environ), environ))
for key in ['REQUEST_METHOD', 'SERVER_NAME', 'SERVER_PORT',
'wsgi.version', 'wsgi.input', 'wsgi.errors',
'wsgi.multithread', 'wsgi.multiprocess',
'wsgi.run_once']:
assert key in environ, (
"Environment missing required key: %r" % key)
for key in ['HTTP_CONTENT_TYPE', 'HTTP_CONTENT_LENGTH']:
assert key not in environ, (
"Environment should not have the key: %s "
"(use %s instead)" % (key, key[5:]))
if 'QUERY_STRING' not in environ:
warnings.warn(
'QUERY_STRING is not in the WSGI environment; the cgi '
'module will use sys.argv when this variable is missing, '
'so application errors are more likely',
WSGIWarning)
for key in environ.keys():
if '.' in key:
# Extension, we don't care about its type
continue
assert type(environ[key]) is StringType, (
"Environmental variable %s is not a string: %r (value: %r)"
% (key, type(environ[key]), environ[key]))
assert type(environ['wsgi.version']) is TupleType, (
"wsgi.version should be a tuple (%r)" % environ['wsgi.version'])
assert environ['wsgi.url_scheme'] in ('http', 'https'), (
"wsgi.url_scheme unknown: %r" % environ['wsgi.url_scheme'])
check_input(environ['wsgi.input'])
check_errors(environ['wsgi.errors'])
# @@: these need filling out:
if environ['REQUEST_METHOD'] not in (
'GET', 'HEAD', 'POST', 'OPTIONS','PUT','DELETE','TRACE'):
warnings.warn(
"Unknown REQUEST_METHOD: %r" % environ['REQUEST_METHOD'],
WSGIWarning)
assert (not environ.get('SCRIPT_NAME')
or environ['SCRIPT_NAME'].startswith('/')), (
"SCRIPT_NAME doesn't start with /: %r" % environ['SCRIPT_NAME'])
assert (not environ.get('PATH_INFO')
or environ['PATH_INFO'].startswith('/')), (
"PATH_INFO doesn't start with /: %r" % environ['PATH_INFO'])
if environ.get('CONTENT_LENGTH'):
assert int(environ['CONTENT_LENGTH']) >= 0, (
"Invalid CONTENT_LENGTH: %r" % environ['CONTENT_LENGTH'])
if not environ.get('SCRIPT_NAME'):
assert environ.has_key('PATH_INFO'), (
"One of SCRIPT_NAME or PATH_INFO are required (PATH_INFO "
"should at least be '/' if SCRIPT_NAME is empty)")
assert environ.get('SCRIPT_NAME') != '/', (
"SCRIPT_NAME cannot be '/'; it should instead be '', and "
"PATH_INFO should be '/'")
def check_input(wsgi_input):
for attr in ['read', 'readline', 'readlines', '__iter__']:
assert hasattr(wsgi_input, attr), (
"wsgi.input (%r) doesn't have the attribute %s"
% (wsgi_input, attr))
def check_errors(wsgi_errors):
for attr in ['flush', 'write', 'writelines']:
assert hasattr(wsgi_errors, attr), (
"wsgi.errors (%r) doesn't have the attribute %s"
% (wsgi_errors, attr))
def check_status(status):
assert type(status) is StringType, (
"Status must be a string (not %r)" % status)
# Implicitly check that we can turn it into an integer:
status_code = status.split(None, 1)[0]
assert len(status_code) == 3, (
"Status codes must be three characters: %r" % status_code)
status_int = int(status_code)
assert status_int >= 100, "Status code is invalid: %r" % status_int
if len(status) < 4 or status[3] != ' ':
warnings.warn(
"The status string (%r) should be a three-digit integer "
"followed by a single space and a status explanation"
% status, WSGIWarning)
def check_headers(headers):
assert type(headers) is ListType, (
"Headers (%r) must be of type list: %r"
% (headers, type(headers)))
header_names = {}
for item in headers:
assert type(item) is TupleType, (
"Individual headers (%r) must be of type tuple: %r"
% (item, type(item)))
assert len(item) == 2
name, value = item
assert name.lower() != 'status', (
"The Status header cannot be used; it conflicts with CGI "
"script, and HTTP status is not given through headers "
"(value: %r)." % value)
header_names[name.lower()] = None
assert '\n' not in name and ':' not in name, (
"Header names may not contain ':' or '\\n': %r" % name)
assert header_re.search(name), "Bad header name: %r" % name
assert not name.endswith('-') and not name.endswith('_'), (
"Names may not end in '-' or '_': %r" % name)
assert not bad_header_value_re.search(value), (
"Bad header value: %r (bad char: %r)"
% (value, bad_header_value_re.search(value).group(0)))
def check_content_type(status, headers):
code = int(status.split(None, 1)[0])
# @@: need one more person to verify this interpretation of RFC 2616
# http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html
NO_MESSAGE_BODY = (204, 304)
NO_MESSAGE_TYPE = (204, 304)
for name, value in headers:
if name.lower() == 'content-type':
if code not in NO_MESSAGE_TYPE:
return
assert 0, (("Content-Type header found in a %s response, "
"which must not return content.") % code)
if code not in NO_MESSAGE_BODY:
assert 0, "No Content-Type header found in headers (%s)" % headers
def check_exc_info(exc_info):
assert exc_info is None or type(exc_info) is type(()), (
"exc_info (%r) is not a tuple: %r" % (exc_info, type(exc_info)))
# More exc_info checks?
def check_iterator(iterator):
# Technically a string is legal, which is why it's a really bad
# idea, because it may cause the response to be returned
# character-by-character
assert not isinstance(iterator, str), (
"You should not return a string as your application iterator, "
"instead return a single-item list containing that string.")
def make_middleware(application, global_conf):
# @@: global_conf should be taken out of the middleware function,
# and isolated here
return middleware(application)
make_middleware.__doc__ = __doc__
__all__ = ['middleware', 'make_middleware']

View File

@@ -0,0 +1,252 @@
"""WSGI Paste wrapper for mod_python. Requires Python 2.2 or greater.
Example httpd.conf section for a Paste app with an ini file::
<Location />
SetHandler python-program
PythonHandler paste.modpython
PythonOption paste.ini /some/location/your/pasteconfig.ini
</Location>
Or if you want to load a WSGI application under /your/homedir in the module
``startup`` and the WSGI app is ``app``::
<Location />
SetHandler python-program
PythonHandler paste.modpython
PythonPath "['/virtual/project/directory'] + sys.path"
PythonOption wsgi.application startup::app
</Location>
If you'd like to use a virtual installation, make sure to add it in the path
like so::
<Location />
SetHandler python-program
PythonHandler paste.modpython
PythonPath "['/virtual/project/directory', '/virtual/lib/python2.4/'] + sys.path"
PythonOption paste.ini /virtual/project/directory/pasteconfig.ini
</Location>
Some WSGI implementations assume that the SCRIPT_NAME environ variable will
always be equal to "the root URL of the app"; Apache probably won't act as
you expect in that case. You can add another PythonOption directive to tell
modpython_gateway to force that behavior:
PythonOption SCRIPT_NAME /mcontrol
Some WSGI applications need to be cleaned up when Apache exits. You can
register a cleanup handler with yet another PythonOption directive:
PythonOption wsgi.cleanup module::function
The module.function will be called with no arguments on server shutdown,
once for each child process or thread.
This module highly based on Robert Brewer's, here:
http://projects.amor.org/misc/svn/modpython_gateway.py
"""
import traceback
try:
from mod_python import apache
except:
pass
from paste.deploy import loadapp
class InputWrapper(object):
def __init__(self, req):
self.req = req
def close(self):
pass
def read(self, size=-1):
return self.req.read(size)
def readline(self, size=-1):
return self.req.readline(size)
def readlines(self, hint=-1):
return self.req.readlines(hint)
def __iter__(self):
line = self.readline()
while line:
yield line
# Notice this won't prefetch the next line; it only
# gets called if the generator is resumed.
line = self.readline()
class ErrorWrapper(object):
def __init__(self, req):
self.req = req
def flush(self):
pass
def write(self, msg):
self.req.log_error(msg)
def writelines(self, seq):
self.write(''.join(seq))
bad_value = ("You must provide a PythonOption '%s', either 'on' or 'off', "
"when running a version of mod_python < 3.1")
class Handler(object):
def __init__(self, req):
self.started = False
options = req.get_options()
# Threading and forking
try:
q = apache.mpm_query
threaded = q(apache.AP_MPMQ_IS_THREADED)
forked = q(apache.AP_MPMQ_IS_FORKED)
except AttributeError:
threaded = options.get('multithread', '').lower()
if threaded == 'on':
threaded = True
elif threaded == 'off':
threaded = False
else:
raise ValueError(bad_value % "multithread")
forked = options.get('multiprocess', '').lower()
if forked == 'on':
forked = True
elif forked == 'off':
forked = False
else:
raise ValueError(bad_value % "multiprocess")
env = self.environ = dict(apache.build_cgi_env(req))
if 'SCRIPT_NAME' in options:
# Override SCRIPT_NAME and PATH_INFO if requested.
env['SCRIPT_NAME'] = options['SCRIPT_NAME']
env['PATH_INFO'] = req.uri[len(options['SCRIPT_NAME']):]
else:
env['SCRIPT_NAME'] = ''
env['PATH_INFO'] = req.uri
env['wsgi.input'] = InputWrapper(req)
env['wsgi.errors'] = ErrorWrapper(req)
env['wsgi.version'] = (1, 0)
env['wsgi.run_once'] = False
if env.get("HTTPS") in ('yes', 'on', '1'):
env['wsgi.url_scheme'] = 'https'
else:
env['wsgi.url_scheme'] = 'http'
env['wsgi.multithread'] = threaded
env['wsgi.multiprocess'] = forked
self.request = req
def run(self, application):
try:
result = application(self.environ, self.start_response)
for data in result:
self.write(data)
if not self.started:
self.request.set_content_length(0)
if hasattr(result, 'close'):
result.close()
except:
traceback.print_exc(None, self.environ['wsgi.errors'])
if not self.started:
self.request.status = 500
self.request.content_type = 'text/plain'
data = "A server error occurred. Please contact the administrator."
self.request.set_content_length(len(data))
self.request.write(data)
def start_response(self, status, headers, exc_info=None):
if exc_info:
try:
if self.started:
raise exc_info[0], exc_info[1], exc_info[2]
finally:
exc_info = None
self.request.status = int(status[:3])
for key, val in headers:
if key.lower() == 'content-length':
self.request.set_content_length(int(val))
elif key.lower() == 'content-type':
self.request.content_type = val
else:
self.request.headers_out.add(key, val)
return self.write
def write(self, data):
if not self.started:
self.started = True
self.request.write(data)
startup = None
cleanup = None
wsgiapps = {}
def handler(req):
options = req.get_options()
# Run a startup function if requested.
global startup
if 'wsgi.startup' in options and not startup:
func = options['wsgi.startup']
if func:
module_name, object_str = func.split('::', 1)
module = __import__(module_name, globals(), locals(), [''])
startup = apache.resolve_object(module, object_str)
startup(req)
# Register a cleanup function if requested.
global cleanup
if 'wsgi.cleanup' in options and not cleanup:
func = options['wsgi.cleanup']
if func:
module_name, object_str = func.split('::', 1)
module = __import__(module_name, globals(), locals(), [''])
cleanup = apache.resolve_object(module, object_str)
def cleaner(data):
cleanup()
try:
# apache.register_cleanup wasn't available until 3.1.4.
apache.register_cleanup(cleaner)
except AttributeError:
req.server.register_cleanup(req, cleaner)
# Import the wsgi 'application' callable and pass it to Handler.run
global wsgiapps
appini = options.get('paste.ini')
app = None
if appini:
if appini not in wsgiapps:
wsgiapps[appini] = loadapp("config:%s" % appini)
app = wsgiapps[appini]
# Import the wsgi 'application' callable and pass it to Handler.run
appwsgi = options.get('wsgi.application')
if appwsgi and not appini:
modname, objname = appwsgi.split('::', 1)
module = __import__(modname, globals(), locals(), [''])
app = getattr(module, objname)
Handler(req).run(app)
# status was set in Handler; always return apache.OK
return apache.OK

View File

@@ -0,0 +1,57 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
"""
We have a pony and/or a unicorn.
"""
from paste.request import construct_url
PONY = """
eJyFkkFuxCAMRfdzCisbJxK2D5D2JpbMrlI3XXQZDt9PCG0ySgcWIMT79rcN0XClUJlZRB9jVmci
FmV19khjgRFl0RzrKmqzvY8lRUWFlXvCrD7UbAQR/17NUvGhypAF9og16vWtkC8DzUayS6pN3/dR
ki0OnpzKjUBFpmlC7zVFRNL1rwoq6PWXXQSnIm9WoTzlM2//ke21o5g/l1ckRhiPbkDZXsKIR7l1
36hF9uMhnRiVjI8UgYjlsIKCrXXpcA9iX5y7zMmtG0fUpW61Ssttipf6cp3WARfkMVoYFryi2a+w
o/2dhW0OXfcMTnmh53oR9egzPs+qkpY9IKxdUVRP5wHO7UDAuI6moA2N+/z4vtc2k8B+AIBimVU=
"""
UNICORN = """
eJyVVD1vhDAM3e9XeAtIxB5P6qlDx0OMXVBzSpZOHdsxP762E0JAnMgZ8Zn37OePAPC60eV1Dl5b
SS7fB6DmQNGhtegpNlPIQS8HmkYGdSqNqDF9wcMYus4TuBYGsZwIPqXfEoNir5K+R3mbzhlR4JMW
eGpikPpn9wHl2sDgEH1270guZwzKDRf3nTztMvfI5r3fJqEmNxdCyISBcWjNgjPG8Egg2hgT3mJi
KBwNvmPB1hbWJ3TwBfMlqdTzxNyDE2H8zOD5HA4KkqJGPVY/TwnxmPA82kdSJNj7zs+R0d1pB+JO
xn2DKgsdxAfFS2pfTSD0Fb6Uzv7dCQSvE5JmZQEQ90vNjBU1GPuGQpCPS8cGo+dQgjIKqxnJTXbw
ucFzPFVIJXtzk6BXKGPnYsKzvFmGx7A0j6Zqvlvk5rETXbMWTGWj0RFc8QNPYVfhJfMMniCPazWJ
lGtPZecIGJWW6oL2hpbWRZEkChe8eg5Wb7xx/MBZBFjxeZPEss+mRQ3Uhc8WQv684seSRO7i3nb4
7HlKUg8sraz47LmXyh8S0somADvoUpoHjGWl+rUkF0H+EIf/gbyyMg58BBk6L634/fkHUCodMw==
"""
class PonyMiddleware(object):
def __init__(self, application):
self.application = application
def __call__(self, environ, start_response):
path_info = environ.get('PATH_INFO', '')
if path_info == '/pony':
url = construct_url(environ, with_query_string=False)
if 'horn' in environ.get('QUERY_STRING', ''):
data = UNICORN
link = 'remove horn!'
else:
data = PONY
url += '?horn'
link = 'add horn!'
msg = data.decode('base64').decode('zlib')
msg = '<pre>%s\n<a href="%s">%s</a></pre>' % (
msg, url, link)
start_response('200 OK', [('content-type', 'text/html')])
return [msg]
else:
return self.application(environ, start_response)
def make_pony(app, global_conf):
"""
Adds pony power to any application, at /pony
"""
return PonyMiddleware(app)

View File

@@ -0,0 +1,222 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
# (c) 2005 Clark C. Evans
# This module is part of the Python Paste Project and is released under
# the MIT License: http://www.opensource.org/licenses/mit-license.php
# This code was written with funding by http://prometheusresearch.com
"""
Upload Progress Monitor
This is a WSGI middleware component which monitors the status of files
being uploaded. It includes a small query application which will return
a list of all files being uploaded by particular session/user.
>>> from paste.httpserver import serve
>>> from paste.urlmap import URLMap
>>> from paste.auth.basic import AuthBasicHandler
>>> from paste.debug.debugapp import SlowConsumer, SimpleApplication
>>> # from paste.progress import *
>>> realm = 'Test Realm'
>>> def authfunc(username, password):
... return username == password
>>> map = URLMap({})
>>> ups = UploadProgressMonitor(map, threshold=1024)
>>> map['/upload'] = SlowConsumer()
>>> map['/simple'] = SimpleApplication()
>>> map['/report'] = UploadProgressReporter(ups)
>>> serve(AuthBasicHandler(ups, realm, authfunc))
serving on...
.. note::
This is experimental, and will change in the future.
"""
import time
from paste.wsgilib import catch_errors
DEFAULT_THRESHOLD = 1024 * 1024 # one megabyte
DEFAULT_TIMEOUT = 60*5 # five minutes
ENVIRON_RECEIVED = 'paste.bytes_received'
REQUEST_STARTED = 'paste.request_started'
REQUEST_FINISHED = 'paste.request_finished'
class _ProgressFile(object):
"""
This is the input-file wrapper used to record the number of
``paste.bytes_received`` for the given request.
"""
def __init__(self, environ, rfile):
self._ProgressFile_environ = environ
self._ProgressFile_rfile = rfile
self.flush = rfile.flush
self.write = rfile.write
self.writelines = rfile.writelines
def __iter__(self):
environ = self._ProgressFile_environ
riter = iter(self._ProgressFile_rfile)
def iterwrap():
for chunk in riter:
environ[ENVIRON_RECEIVED] += len(chunk)
yield chunk
return iter(iterwrap)
def read(self, size=-1):
chunk = self._ProgressFile_rfile.read(size)
self._ProgressFile_environ[ENVIRON_RECEIVED] += len(chunk)
return chunk
def readline(self):
chunk = self._ProgressFile_rfile.readline()
self._ProgressFile_environ[ENVIRON_RECEIVED] += len(chunk)
return chunk
def readlines(self, hint=None):
chunk = self._ProgressFile_rfile.readlines(hint)
self._ProgressFile_environ[ENVIRON_RECEIVED] += len(chunk)
return chunk
class UploadProgressMonitor(object):
"""
monitors and reports on the status of uploads in progress
Parameters:
``application``
This is the next application in the WSGI stack.
``threshold``
This is the size in bytes that is needed for the
upload to be included in the monitor.
``timeout``
This is the amount of time (in seconds) that a upload
remains in the monitor after it has finished.
Methods:
``uploads()``
This returns a list of ``environ`` dict objects for each
upload being currently monitored, or finished but whose time
has not yet expired.
For each request ``environ`` that is monitored, there are several
variables that are stored:
``paste.bytes_received``
This is the total number of bytes received for the given
request; it can be compared with ``CONTENT_LENGTH`` to
build a percentage complete. This is an integer value.
``paste.request_started``
This is the time (in seconds) when the request was started
as obtained from ``time.time()``. One would want to format
this for presentation to the user, if necessary.
``paste.request_finished``
This is the time (in seconds) when the request was finished,
canceled, or otherwise disconnected. This is None while
the given upload is still in-progress.
TODO: turn monitor into a queue and purge queue of finished
requests that have passed the timeout period.
"""
def __init__(self, application, threshold=None, timeout=None):
self.application = application
self.threshold = threshold or DEFAULT_THRESHOLD
self.timeout = timeout or DEFAULT_TIMEOUT
self.monitor = []
def __call__(self, environ, start_response):
length = environ.get('CONTENT_LENGTH', 0)
if length and int(length) > self.threshold:
# replace input file object
self.monitor.append(environ)
environ[ENVIRON_RECEIVED] = 0
environ[REQUEST_STARTED] = time.time()
environ[REQUEST_FINISHED] = None
environ['wsgi.input'] = \
_ProgressFile(environ, environ['wsgi.input'])
def finalizer(exc_info=None):
environ[REQUEST_FINISHED] = time.time()
return catch_errors(self.application, environ,
start_response, finalizer, finalizer)
return self.application(environ, start_response)
def uploads(self):
return self.monitor
class UploadProgressReporter(object):
"""
reports on the progress of uploads for a given user
This reporter returns a JSON file (for use in AJAX) listing the
uploads in progress for the given user. By default, this reporter
uses the ``REMOTE_USER`` environment to compare between the current
request and uploads in-progress. If they match, then a response
record is formed.
``match()``
This member function can be overriden to provide alternative
matching criteria. It takes two environments, the first
is the current request, the second is a current upload.
``report()``
This member function takes an environment and builds a
``dict`` that will be used to create a JSON mapping for
the given upload. By default, this just includes the
percent complete and the request url.
"""
def __init__(self, monitor):
self.monitor = monitor
def match(self, search_environ, upload_environ):
if search_environ.get('REMOTE_USER', None) == \
upload_environ.get('REMOTE_USER', 0):
return True
return False
def report(self, environ):
retval = { 'started': time.strftime("%Y-%m-%d %H:%M:%S",
time.gmtime(environ[REQUEST_STARTED])),
'finished': '',
'content_length': environ.get('CONTENT_LENGTH'),
'bytes_received': environ[ENVIRON_RECEIVED],
'path_info': environ.get('PATH_INFO',''),
'query_string': environ.get('QUERY_STRING','')}
finished = environ[REQUEST_FINISHED]
if finished:
retval['finished'] = time.strftime("%Y:%m:%d %H:%M:%S",
time.gmtime(finished))
return retval
def __call__(self, environ, start_response):
body = []
for map in [self.report(env) for env in self.monitor.uploads()
if self.match(environ, env)]:
parts = []
for k, v in map.items():
v = str(v).replace("\\", "\\\\").replace('"', '\\"')
parts.append('%s: "%s"' % (k, v))
body.append("{ %s }" % ", ".join(parts))
body = "[ %s ]" % ", ".join(body)
start_response("200 OK", [('Content-Type', 'text/plain'),
('Content-Length', len(body))])
return [body]
__all__ = ['UploadProgressMonitor', 'UploadProgressReporter']
if "__main__" == __name__:
import doctest
doctest.testmod(optionflags=doctest.ELLIPSIS)

View File

@@ -0,0 +1,283 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
"""
An application that proxies WSGI requests to a remote server.
TODO:
* Send ``Via`` header? It's not clear to me this is a Via in the
style of a typical proxy.
* Other headers or metadata? I put in X-Forwarded-For, but that's it.
* Signed data of non-HTTP keys? This would be for things like
REMOTE_USER.
* Something to indicate what the original URL was? The original host,
scheme, and base path.
* Rewriting ``Location`` headers? mod_proxy does this.
* Rewriting body? (Probably not on this one -- that can be done with
a different middleware that wraps this middleware)
* Example::
use = egg:Paste#proxy
address = http://server3:8680/exist/rest/db/orgs/sch/config/
allowed_request_methods = GET
"""
import httplib
import urlparse
import urllib
from paste import httpexceptions
from paste.util.converters import aslist
# Remove these headers from response (specify lower case header
# names):
filtered_headers = (
'transfer-encoding',
'connection',
'keep-alive',
'proxy-authenticate',
'proxy-authorization',
'te',
'trailers',
'upgrade',
)
class Proxy(object):
def __init__(self, address, allowed_request_methods=(),
suppress_http_headers=()):
self.address = address
self.parsed = urlparse.urlsplit(address)
self.scheme = self.parsed[0].lower()
self.host = self.parsed[1]
self.path = self.parsed[2]
self.allowed_request_methods = [
x.lower() for x in allowed_request_methods if x]
self.suppress_http_headers = [
x.lower() for x in suppress_http_headers if x]
def __call__(self, environ, start_response):
if (self.allowed_request_methods and
environ['REQUEST_METHOD'].lower() not in self.allowed_request_methods):
return httpexceptions.HTTPBadRequest("Disallowed")(environ, start_response)
if self.scheme == 'http':
ConnClass = httplib.HTTPConnection
elif self.scheme == 'https':
ConnClass = httplib.HTTPSConnection
else:
raise ValueError(
"Unknown scheme for %r: %r" % (self.address, self.scheme))
conn = ConnClass(self.host)
headers = {}
for key, value in environ.items():
if key.startswith('HTTP_'):
key = key[5:].lower().replace('_', '-')
if key == 'host' or key in self.suppress_http_headers:
continue
headers[key] = value
headers['host'] = self.host
if 'REMOTE_ADDR' in environ:
headers['x-forwarded-for'] = environ['REMOTE_ADDR']
if environ.get('CONTENT_TYPE'):
headers['content-type'] = environ['CONTENT_TYPE']
if environ.get('CONTENT_LENGTH'):
if environ['CONTENT_LENGTH'] == '-1':
# This is a special case, where the content length is basically undetermined
body = environ['wsgi.input'].read(-1)
headers['content-length'] = str(len(body))
else:
headers['content-length'] = environ['CONTENT_LENGTH']
length = int(environ['CONTENT_LENGTH'])
body = environ['wsgi.input'].read(length)
else:
body = ''
path_info = urllib.quote(environ['PATH_INFO'])
if self.path:
request_path = path_info
if request_path and request_path[0] == '/':
request_path = request_path[1:]
path = urlparse.urljoin(self.path, request_path)
else:
path = path_info
if environ.get('QUERY_STRING'):
path += '?' + environ['QUERY_STRING']
conn.request(environ['REQUEST_METHOD'],
path,
body, headers)
res = conn.getresponse()
headers_out = parse_headers(res.msg)
status = '%s %s' % (res.status, res.reason)
start_response(status, headers_out)
# @@: Default?
length = res.getheader('content-length')
if length is not None:
body = res.read(int(length))
else:
body = res.read()
conn.close()
return [body]
def make_proxy(global_conf, address, allowed_request_methods="",
suppress_http_headers=""):
"""
Make a WSGI application that proxies to another address:
``address``
the full URL ending with a trailing ``/``
``allowed_request_methods``:
a space seperated list of request methods (e.g., ``GET POST``)
``suppress_http_headers``
a space seperated list of http headers (lower case, without
the leading ``http_``) that should not be passed on to target
host
"""
allowed_request_methods = aslist(allowed_request_methods)
suppress_http_headers = aslist(suppress_http_headers)
return Proxy(
address,
allowed_request_methods=allowed_request_methods,
suppress_http_headers=suppress_http_headers)
class TransparentProxy(object):
"""
A proxy that sends the request just as it was given, including
respecting HTTP_HOST, wsgi.url_scheme, etc.
This is a way of translating WSGI requests directly to real HTTP
requests. All information goes in the environment; modify it to
modify the way the request is made.
If you specify ``force_host`` (and optionally ``force_scheme``)
then HTTP_HOST won't be used to determine where to connect to;
instead a specific host will be connected to, but the ``Host``
header in the request will remain intact.
"""
def __init__(self, force_host=None,
force_scheme='http'):
self.force_host = force_host
self.force_scheme = force_scheme
def __repr__(self):
return '<%s %s force_host=%r force_scheme=%r>' % (
self.__class__.__name__,
hex(id(self)),
self.force_host, self.force_scheme)
def __call__(self, environ, start_response):
scheme = environ['wsgi.url_scheme']
if self.force_host is None:
conn_scheme = scheme
else:
conn_scheme = self.force_scheme
if conn_scheme == 'http':
ConnClass = httplib.HTTPConnection
elif conn_scheme == 'https':
ConnClass = httplib.HTTPSConnection
else:
raise ValueError(
"Unknown scheme %r" % scheme)
if 'HTTP_HOST' not in environ:
raise ValueError(
"WSGI environ must contain an HTTP_HOST key")
host = environ['HTTP_HOST']
if self.force_host is None:
conn_host = host
else:
conn_host = self.force_host
conn = ConnClass(conn_host)
headers = {}
for key, value in environ.items():
if key.startswith('HTTP_'):
key = key[5:].lower().replace('_', '-')
headers[key] = value
headers['host'] = host
if 'REMOTE_ADDR' in environ and 'HTTP_X_FORWARDED_FOR' not in environ:
headers['x-forwarded-for'] = environ['REMOTE_ADDR']
if environ.get('CONTENT_TYPE'):
headers['content-type'] = environ['CONTENT_TYPE']
if environ.get('CONTENT_LENGTH'):
length = int(environ['CONTENT_LENGTH'])
body = environ['wsgi.input'].read(length)
if length == -1:
environ['CONTENT_LENGTH'] = str(len(body))
elif 'CONTENT_LENGTH' not in environ:
body = ''
length = 0
else:
body = ''
length = 0
path = (environ.get('SCRIPT_NAME', '')
+ environ.get('PATH_INFO', ''))
path = urllib.quote(path)
if 'QUERY_STRING' in environ:
path += '?' + environ['QUERY_STRING']
conn.request(environ['REQUEST_METHOD'],
path, body, headers)
res = conn.getresponse()
headers_out = parse_headers(res.msg)
status = '%s %s' % (res.status, res.reason)
start_response(status, headers_out)
# @@: Default?
length = res.getheader('content-length')
if length is not None:
body = res.read(int(length))
else:
body = res.read()
conn.close()
return [body]
def parse_headers(message):
"""
Turn a Message object into a list of WSGI-style headers.
"""
headers_out = []
for full_header in message.headers:
if not full_header:
# Shouldn't happen, but we'll just ignore
continue
if full_header[0].isspace():
# Continuation line, add to the last header
if not headers_out:
raise ValueError(
"First header starts with a space (%r)" % full_header)
last_header, last_value = headers_out.pop()
value = last_value + ' ' + full_header.strip()
headers_out.append((last_header, value))
continue
try:
header, value = full_header.split(':', 1)
except:
raise ValueError("Invalid header: %r" % full_header)
value = value.strip()
if header.lower() not in filtered_headers:
headers_out.append((header, value))
return headers_out
def make_transparent_proxy(
global_conf, force_host=None, force_scheme='http'):
"""
Create a proxy that connects to a specific host, but does
absolutely no other filtering, including the Host header.
"""
return TransparentProxy(force_host=force_host,
force_scheme=force_scheme)

View File

@@ -0,0 +1,405 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
"""
Middleware to make internal requests and forward requests internally.
When applied, several keys are added to the environment that will allow
you to trigger recursive redirects and forwards.
paste.recursive.include:
When you call
``environ['paste.recursive.include'](new_path_info)`` a response
will be returned. The response has a ``body`` attribute, a
``status`` attribute, and a ``headers`` attribute.
paste.recursive.script_name:
The ``SCRIPT_NAME`` at the point that recursive lives. Only
paths underneath this path can be redirected to.
paste.recursive.old_path_info:
A list of previous ``PATH_INFO`` values from previous redirects.
Raise ``ForwardRequestException(new_path_info)`` to do a forward
(aborting the current request).
"""
from cStringIO import StringIO
import warnings
__all__ = ['RecursiveMiddleware']
__pudge_all__ = ['RecursiveMiddleware', 'ForwardRequestException']
class RecursionLoop(AssertionError):
# Subclasses AssertionError for legacy reasons
"""Raised when a recursion enters into a loop"""
class CheckForRecursionMiddleware(object):
def __init__(self, app, env):
self.app = app
self.env = env
def __call__(self, environ, start_response):
path_info = environ.get('PATH_INFO','')
if path_info in self.env.get(
'paste.recursive.old_path_info', []):
raise RecursionLoop(
"Forwarding loop detected; %r visited twice (internal "
"redirect path: %s)"
% (path_info, self.env['paste.recursive.old_path_info']))
old_path_info = self.env.setdefault('paste.recursive.old_path_info', [])
old_path_info.append(self.env.get('PATH_INFO', ''))
return self.app(environ, start_response)
class RecursiveMiddleware(object):
"""
A WSGI middleware that allows for recursive and forwarded calls.
All these calls go to the same 'application', but presumably that
application acts differently with different URLs. The forwarded
URLs must be relative to this container.
Interface is entirely through the ``paste.recursive.forward`` and
``paste.recursive.include`` environmental keys.
"""
def __init__(self, application, global_conf=None):
self.application = application
def __call__(self, environ, start_response):
environ['paste.recursive.forward'] = Forwarder(
self.application,
environ,
start_response)
environ['paste.recursive.include'] = Includer(
self.application,
environ,
start_response)
environ['paste.recursive.include_app_iter'] = IncluderAppIter(
self.application,
environ,
start_response)
my_script_name = environ.get('SCRIPT_NAME', '')
environ['paste.recursive.script_name'] = my_script_name
try:
return self.application(environ, start_response)
except ForwardRequestException, e:
middleware = CheckForRecursionMiddleware(
e.factory(self), environ)
return middleware(environ, start_response)
class ForwardRequestException(Exception):
"""
Used to signal that a request should be forwarded to a different location.
``url``
The URL to forward to starting with a ``/`` and relative to
``RecursiveMiddleware``. URL fragments can also contain query strings
so ``/error?code=404`` would be a valid URL fragment.
``environ``
An altertative WSGI environment dictionary to use for the forwarded
request. If specified is used *instead* of the ``url_fragment``
``factory``
If specifed ``factory`` is used instead of ``url`` or ``environ``.
``factory`` is a callable that takes a WSGI application object
as the first argument and returns an initialised WSGI middleware
which can alter the forwarded response.
Basic usage (must have ``RecursiveMiddleware`` present) :
.. code-block:: python
from paste.recursive import ForwardRequestException
def app(environ, start_response):
if environ['PATH_INFO'] == '/hello':
start_response("200 OK", [('Content-type', 'text/plain')])
return ['Hello World!']
elif environ['PATH_INFO'] == '/error':
start_response("404 Not Found", [('Content-type', 'text/plain')])
return ['Page not found']
else:
raise ForwardRequestException('/error')
from paste.recursive import RecursiveMiddleware
app = RecursiveMiddleware(app)
If you ran this application and visited ``/hello`` you would get a
``Hello World!`` message. If you ran the application and visited
``/not_found`` a ``ForwardRequestException`` would be raised and the caught
by the ``RecursiveMiddleware``. The ``RecursiveMiddleware`` would then
return the headers and response from the ``/error`` URL but would display
a ``404 Not found`` status message.
You could also specify an ``environ`` dictionary instead of a url. Using
the same example as before:
.. code-block:: python
def app(environ, start_response):
... same as previous example ...
else:
new_environ = environ.copy()
new_environ['PATH_INFO'] = '/error'
raise ForwardRequestException(environ=new_environ)
Finally, if you want complete control over every aspect of the forward you
can specify a middleware factory. For example to keep the old status code
but use the headers and resposne body from the forwarded response you might
do this:
.. code-block:: python
from paste.recursive import ForwardRequestException
from paste.recursive import RecursiveMiddleware
from paste.errordocument import StatusKeeper
def app(environ, start_response):
if environ['PATH_INFO'] == '/hello':
start_response("200 OK", [('Content-type', 'text/plain')])
return ['Hello World!']
elif environ['PATH_INFO'] == '/error':
start_response("404 Not Found", [('Content-type', 'text/plain')])
return ['Page not found']
else:
def factory(app):
return StatusKeeper(app, status='404 Not Found', url='/error')
raise ForwardRequestException(factory=factory)
app = RecursiveMiddleware(app)
"""
def __init__(
self,
url=None,
environ={},
factory=None,
path_info=None):
# Check no incompatible options have been chosen
if factory and url:
raise TypeError(
'You cannot specify factory and a url in '
'ForwardRequestException')
elif factory and environ:
raise TypeError(
'You cannot specify factory and environ in '
'ForwardRequestException')
if url and environ:
raise TypeError(
'You cannot specify environ and url in '
'ForwardRequestException')
# set the path_info or warn about its use.
if path_info:
if not url:
warnings.warn(
"ForwardRequestException(path_info=...) has been deprecated; please "
"use ForwardRequestException(url=...)",
DeprecationWarning, 2)
else:
raise TypeError('You cannot use url and path_info in ForwardRequestException')
self.path_info = path_info
# If the url can be treated as a path_info do that
if url and not '?' in str(url):
self.path_info = url
# Base middleware
class ForwardRequestExceptionMiddleware(object):
def __init__(self, app):
self.app = app
# Otherwise construct the appropriate middleware factory
if hasattr(self, 'path_info'):
p = self.path_info
def factory_(app):
class PathInfoForward(ForwardRequestExceptionMiddleware):
def __call__(self, environ, start_response):
environ['PATH_INFO'] = p
return self.app(environ, start_response)
return PathInfoForward(app)
self.factory = factory_
elif url:
def factory_(app):
class URLForward(ForwardRequestExceptionMiddleware):
def __call__(self, environ, start_response):
environ['PATH_INFO'] = url.split('?')[0]
environ['QUERY_STRING'] = url.split('?')[1]
return self.app(environ, start_response)
return URLForward(app)
self.factory = factory_
elif environ:
def factory_(app):
class EnvironForward(ForwardRequestExceptionMiddleware):
def __call__(self, environ_, start_response):
return self.app(environ, start_response)
return EnvironForward(app)
self.factory = factory_
else:
self.factory = factory
class Recursive(object):
def __init__(self, application, environ, start_response):
self.application = application
self.original_environ = environ.copy()
self.previous_environ = environ
self.start_response = start_response
def __call__(self, path, extra_environ=None):
"""
`extra_environ` is an optional dictionary that is also added
to the forwarded request. E.g., ``{'HTTP_HOST': 'new.host'}``
could be used to forward to a different virtual host.
"""
environ = self.original_environ.copy()
if extra_environ:
environ.update(extra_environ)
environ['paste.recursive.previous_environ'] = self.previous_environ
base_path = self.original_environ.get('SCRIPT_NAME')
if path.startswith('/'):
assert path.startswith(base_path), (
"You can only forward requests to resources under the "
"path %r (not %r)" % (base_path, path))
path = path[len(base_path)+1:]
assert not path.startswith('/')
path_info = '/' + path
environ['PATH_INFO'] = path_info
environ['REQUEST_METHOD'] = 'GET'
environ['CONTENT_LENGTH'] = '0'
environ['CONTENT_TYPE'] = ''
environ['wsgi.input'] = StringIO('')
return self.activate(environ)
def activate(self, environ):
raise NotImplementedError
def __repr__(self):
return '<%s.%s from %s>' % (
self.__class__.__module__,
self.__class__.__name__,
self.original_environ.get('SCRIPT_NAME') or '/')
class Forwarder(Recursive):
"""
The forwarder will try to restart the request, except with
the new `path` (replacing ``PATH_INFO`` in the request).
It must not be called after and headers have been returned.
It returns an iterator that must be returned back up the call
stack, so it must be used like:
.. code-block:: python
return environ['paste.recursive.forward'](path)
Meaningful transformations cannot be done, since headers are
sent directly to the server and cannot be inspected or
rewritten.
"""
def activate(self, environ):
warnings.warn(
"recursive.Forwarder has been deprecated; please use "
"ForwardRequestException",
DeprecationWarning, 2)
return self.application(environ, self.start_response)
class Includer(Recursive):
"""
Starts another request with the given path and adding or
overwriting any values in the `extra_environ` dictionary.
Returns an IncludeResponse object.
"""
def activate(self, environ):
response = IncludedResponse()
def start_response(status, headers, exc_info=None):
if exc_info:
raise exc_info[0], exc_info[1], exc_info[2]
response.status = status
response.headers = headers
return response.write
app_iter = self.application(environ, start_response)
try:
for s in app_iter:
response.write(s)
finally:
if hasattr(app_iter, 'close'):
app_iter.close()
response.close()
return response
class IncludedResponse(object):
def __init__(self):
self.headers = None
self.status = None
self.output = StringIO()
self.str = None
def close(self):
self.str = self.output.getvalue()
self.output.close()
self.output = None
def write(self, s):
assert self.output is not None, (
"This response has already been closed and no further data "
"can be written.")
self.output.write(s)
def __str__(self):
return self.body
def body__get(self):
if self.str is None:
return self.output.getvalue()
else:
return self.str
body = property(body__get)
class IncluderAppIter(Recursive):
"""
Like Includer, but just stores the app_iter response
(be sure to call close on the response!)
"""
def activate(self, environ):
response = IncludedAppIterResponse()
def start_response(status, headers, exc_info=None):
if exc_info:
raise exc_info[0], exc_info[1], exc_info[2]
response.status = status
response.headers = headers
return response.write
app_iter = self.application(environ, start_response)
response.app_iter = app_iter
return response
class IncludedAppIterResponse(object):
def __init__(self):
self.status = None
self.headers = None
self.accumulated = []
self.app_iter = None
self._closed = False
def close(self):
assert not self._closed, (
"Tried to close twice")
if hasattr(self.app_iter, 'close'):
self.app_iter.close()
def write(self, s):
self.accumulated.append
def make_recursive_middleware(app, global_conf):
return RecursiveMiddleware(app)
make_recursive_middleware.__doc__ = __doc__

View File

@@ -0,0 +1,581 @@
# (c) 2005 Ben Bangert
# This module is part of the Python Paste Project and is released under
# the MIT License: http://www.opensource.org/licenses/mit-license.php
"""Registry for handling request-local module globals sanely
Dealing with module globals in a thread-safe way is good if your
application is the sole responder in a thread, however that approach fails
to properly account for various scenarios that occur with WSGI applications
and middleware.
What is actually needed in the case where a module global is desired that
is always set properly depending on the current request, is a stacked
thread-local object. Such an object is popped or pushed during the request
cycle so that it properly represents the object that should be active for
the current request.
To make it easy to deal with such variables, this module provides a special
StackedObjectProxy class which you can instantiate and attach to your
module where you'd like others to access it. The object you'd like this to
actually "be" during the request is then registered with the
RegistryManager middleware, which ensures that for the scope of the current
WSGI application everything will work properly.
Example:
.. code-block:: python
#yourpackage/__init__.py
from paste.registry import RegistryManager, StackedObjectProxy
myglobal = StackedObjectProxy()
#wsgi app stack
app = RegistryManager(yourapp)
#inside your wsgi app
class yourapp(object):
def __call__(self, environ, start_response):
obj = someobject # The request-local object you want to access
# via yourpackage.myglobal
if environ.has_key('paste.registry'):
environ['paste.registry'].register(myglobal, obj)
You will then be able to import yourpackage anywhere in your WSGI app or in
the calling stack below it and be assured that it is using the object you
registered with Registry.
RegistryManager can be in the WSGI stack multiple times, each time it
appears it registers a new request context.
Performance
===========
The overhead of the proxy object is very minimal, however if you are using
proxy objects extensively (Thousands of accesses per request or more), there
are some ways to avoid them. A proxy object runs approximately 3-20x slower
than direct access to the object, this is rarely your performance bottleneck
when developing web applications.
Should you be developing a system which may be accessing the proxy object
thousands of times per request, the performance of the proxy will start to
become more noticeable. In that circumstance, the problem can be avoided by
getting at the actual object via the proxy with the ``_current_obj`` function:
.. code-block:: python
#sessions.py
Session = StackedObjectProxy()
# ... initialization code, etc.
# somemodule.py
import sessions
def somefunc():
session = sessions.Session._current_obj()
# ... tons of session access
This way the proxy is used only once to retrieve the object for the current
context and the overhead is minimized while still making it easy to access
the underlying object. The ``_current_obj`` function is preceded by an
underscore to more likely avoid clashing with the contained object's
attributes.
**NOTE:** This is *highly* unlikely to be an issue in the vast majority of
cases, and requires incredibly large amounts of proxy object access before
one should consider the proxy object to be causing slow-downs. This section
is provided solely in the extremely rare case that it is an issue so that a
quick way to work around it is documented.
"""
import sys
import paste.util.threadinglocal as threadinglocal
__all__ = ['StackedObjectProxy', 'RegistryManager', 'StackedObjectRestorer',
'restorer']
class NoDefault(object): pass
class StackedObjectProxy(object):
"""Track an object instance internally using a stack
The StackedObjectProxy proxies access to an object internally using a
stacked thread-local. This makes it safe for complex WSGI environments
where access to the object may be desired in multiple places without
having to pass the actual object around.
New objects are added to the top of the stack with _push_object while
objects can be removed with _pop_object.
"""
def __init__(self, default=NoDefault, name="Default"):
"""Create a new StackedObjectProxy
If a default is given, its used in every thread if no other object
has been pushed on.
"""
self.__dict__['____name__'] = name
self.__dict__['____local__'] = threadinglocal.local()
if default is not NoDefault:
self.__dict__['____default_object__'] = default
def __dir__(self):
"""Return a list of the StackedObjectProxy's and proxied
object's (if one exists) names.
"""
dir_list = dir(self.__class__) + self.__dict__.keys()
try:
dir_list.extend(dir(self._current_obj()))
except TypeError:
pass
dir_list.sort()
return dir_list
def __getattr__(self, attr):
return getattr(self._current_obj(), attr)
def __setattr__(self, attr, value):
setattr(self._current_obj(), attr, value)
def __delattr__(self, name):
delattr(self._current_obj(), name)
def __getitem__(self, key):
return self._current_obj()[key]
def __setitem__(self, key, value):
self._current_obj()[key] = value
def __delitem__(self, key):
del self._current_obj()[key]
def __call__(self, *args, **kw):
return self._current_obj()(*args, **kw)
def __repr__(self):
try:
return repr(self._current_obj())
except (TypeError, AttributeError):
return '<%s.%s object at 0x%x>' % (self.__class__.__module__,
self.__class__.__name__,
id(self))
def __iter__(self):
return iter(self._current_obj())
def __len__(self):
return len(self._current_obj())
def __contains__(self, key):
return key in self._current_obj()
def __nonzero__(self):
return bool(self._current_obj())
def _current_obj(self):
"""Returns the current active object being proxied to
In the event that no object was pushed, the default object if
provided will be used. Otherwise, a TypeError will be raised.
"""
try:
objects = self.____local__.objects
except AttributeError:
objects = None
if objects:
return objects[-1]
else:
obj = self.__dict__.get('____default_object__', NoDefault)
if obj is not NoDefault:
return obj
else:
raise TypeError(
'No object (name: %s) has been registered for this '
'thread' % self.____name__)
def _push_object(self, obj):
"""Make ``obj`` the active object for this thread-local.
This should be used like:
.. code-block:: python
obj = yourobject()
module.glob = StackedObjectProxy()
module.glob._push_object(obj)
try:
... do stuff ...
finally:
module.glob._pop_object(conf)
"""
try:
self.____local__.objects.append(obj)
except AttributeError:
self.____local__.objects = []
self.____local__.objects.append(obj)
def _pop_object(self, obj=None):
"""Remove a thread-local object.
If ``obj`` is given, it is checked against the popped object and an
error is emitted if they don't match.
"""
try:
popped = self.____local__.objects.pop()
if obj and popped is not obj:
raise AssertionError(
'The object popped (%s) is not the same as the object '
'expected (%s)' % (popped, obj))
except AttributeError:
raise AssertionError('No object has been registered for this thread')
def _object_stack(self):
"""Returns all of the objects stacked in this container
(Might return [] if there are none)
"""
try:
try:
objs = self.____local__.objects
except AttributeError:
return []
return objs[:]
except AssertionError:
return []
# The following methods will be swapped for their original versions by
# StackedObjectRestorer when restoration is enabled. The original
# functions (e.g. _current_obj) will be available at _current_obj_orig
def _current_obj_restoration(self):
request_id = restorer.in_restoration()
if request_id:
return restorer.get_saved_proxied_obj(self, request_id)
return self._current_obj_orig()
_current_obj_restoration.__doc__ = \
('%s\n(StackedObjectRestorer restoration enabled)' % \
_current_obj.__doc__)
def _push_object_restoration(self, obj):
if not restorer.in_restoration():
self._push_object_orig(obj)
_push_object_restoration.__doc__ = \
('%s\n(StackedObjectRestorer restoration enabled)' % \
_push_object.__doc__)
def _pop_object_restoration(self, obj=None):
if not restorer.in_restoration():
self._pop_object_orig(obj)
_pop_object_restoration.__doc__ = \
('%s\n(StackedObjectRestorer restoration enabled)' % \
_pop_object.__doc__)
class Registry(object):
"""Track objects and stacked object proxies for removal
The Registry object is instantiated a single time for the request no
matter how many times the RegistryManager is used in a WSGI stack. Each
RegistryManager must call ``prepare`` before continuing the call to
start a new context for object registering.
Each context is tracked with a dict inside a list. The last list
element is the currently executing context. Each context dict is keyed
by the id of the StackedObjectProxy instance being proxied, the value
is a tuple of the StackedObjectProxy instance and the object being
tracked.
"""
def __init__(self):
"""Create a new Registry object
``prepare`` must still be called before this Registry object can be
used to register objects.
"""
self.reglist = []
def prepare(self):
"""Used to create a new registry context
Anytime a new RegistryManager is called, ``prepare`` needs to be
called on the existing Registry object. This sets up a new context
for registering objects.
"""
self.reglist.append({})
def register(self, stacked, obj):
"""Register an object with a StackedObjectProxy"""
myreglist = self.reglist[-1]
stacked_id = id(stacked)
if stacked_id in myreglist:
stacked._pop_object(myreglist[stacked_id][1])
del myreglist[stacked_id]
stacked._push_object(obj)
myreglist[stacked_id] = (stacked, obj)
def multiregister(self, stacklist):
"""Register a list of tuples
Similar call semantics as register, except this registers
multiple objects at once.
Example::
registry.multiregister([(sop, obj), (anothersop, anotherobj)])
"""
myreglist = self.reglist[-1]
for stacked, obj in stacklist:
stacked_id = id(stacked)
if stacked_id in myreglist:
stacked._pop_object(myreglist[stacked_id][1])
del myreglist[stacked_id]
stacked._push_object(obj)
myreglist[stacked_id] = (stacked, obj)
# Replace now does the same thing as register
replace = register
def cleanup(self):
"""Remove all objects from all StackedObjectProxy instances that
were tracked at this Registry context"""
for stacked, obj in self.reglist[-1].itervalues():
stacked._pop_object(obj)
self.reglist.pop()
class RegistryManager(object):
"""Creates and maintains a Registry context
RegistryManager creates a new registry context for the registration of
StackedObjectProxy instances. Multiple RegistryManager's can be in a
WSGI stack and will manage the context so that the StackedObjectProxies
always proxy to the proper object.
The object being registered can be any object sub-class, list, or dict.
Registering objects is done inside a WSGI application under the
RegistryManager instance, using the ``environ['paste.registry']``
object which is a Registry instance.
"""
def __init__(self, application, streaming=False):
self.application = application
self.streaming = streaming
def __call__(self, environ, start_response):
app_iter = None
reg = environ.setdefault('paste.registry', Registry())
reg.prepare()
if self.streaming:
return self.streaming_iter(reg, environ, start_response)
try:
app_iter = self.application(environ, start_response)
except Exception, e:
# Regardless of if the content is an iterable, generator, list
# or tuple, we clean-up right now. If its an iterable/generator
# care should be used to ensure the generator has its own ref
# to the actual object
if environ.get('paste.evalexception'):
# EvalException is present in the WSGI stack
expected = False
for expect in environ.get('paste.expected_exceptions', []):
if isinstance(e, expect):
expected = True
if not expected:
# An unexpected exception: save state for EvalException
restorer.save_registry_state(environ)
reg.cleanup()
raise
except:
# Save state for EvalException if it's present
if environ.get('paste.evalexception'):
restorer.save_registry_state(environ)
reg.cleanup()
raise
else:
reg.cleanup()
return app_iter
def streaming_iter(self, reg, environ, start_response):
try:
for item in self.application(environ, start_response):
yield item
except Exception, e:
# Regardless of if the content is an iterable, generator, list
# or tuple, we clean-up right now. If its an iterable/generator
# care should be used to ensure the generator has its own ref
# to the actual object
if environ.get('paste.evalexception'):
# EvalException is present in the WSGI stack
expected = False
for expect in environ.get('paste.expected_exceptions', []):
if isinstance(e, expect):
expected = True
if not expected:
# An unexpected exception: save state for EvalException
restorer.save_registry_state(environ)
reg.cleanup()
raise
except:
# Save state for EvalException if it's present
if environ.get('paste.evalexception'):
restorer.save_registry_state(environ)
reg.cleanup()
raise
else:
reg.cleanup()
class StackedObjectRestorer(object):
"""Track StackedObjectProxies and their proxied objects for automatic
restoration within EvalException's interactive debugger.
An instance of this class tracks all StackedObjectProxy state in existence
when unexpected exceptions are raised by WSGI applications housed by
EvalException and RegistryManager. Like EvalException, this information is
stored for the life of the process.
When an unexpected exception occurs and EvalException is present in the
WSGI stack, save_registry_state is intended to be called to store the
Registry state and enable automatic restoration on all currently registered
StackedObjectProxies.
With restoration enabled, those StackedObjectProxies' _current_obj
(overwritten by _current_obj_restoration) method's strategy is modified:
it will return its appropriate proxied object from the restorer when
a restoration context is active in the current thread.
The StackedObjectProxies' _push/pop_object methods strategies are also
changed: they no-op when a restoration context is active in the current
thread (because the pushing/popping work is all handled by the
Registry/restorer).
The request's Registry objects' reglists are restored from the restorer
when a restoration context begins, enabling the Registry methods to work
while their changes are tracked by the restorer.
The overhead of enabling restoration is negligible (another threadlocal
access for the changed StackedObjectProxy methods) for normal use outside
of a restoration context, but worth mentioning when combined with
StackedObjectProxies normal overhead. Once enabled it does not turn off,
however:
o Enabling restoration only occurs after an unexpected exception is
detected. The server is likely to be restarted shortly after the exception
is raised to fix the cause
o StackedObjectRestorer is only enabled when EvalException is enabled (not
on a production server) and RegistryManager exists in the middleware
stack"""
def __init__(self):
# Registries and their saved reglists by request_id
self.saved_registry_states = {}
self.restoration_context_id = threadinglocal.local()
def save_registry_state(self, environ):
"""Save the state of this request's Registry (if it hasn't already been
saved) to the saved_registry_states dict, keyed by the request's unique
identifier"""
registry = environ.get('paste.registry')
if not registry or not len(registry.reglist) or \
self.get_request_id(environ) in self.saved_registry_states:
# No Registry, no state to save, or this request's state has
# already been saved
return
self.saved_registry_states[self.get_request_id(environ)] = \
(registry, registry.reglist[:])
# Tweak the StackedObjectProxies we want to save state for -- change
# their methods to act differently when a restoration context is active
# in the current thread
for reglist in registry.reglist:
for stacked, obj in reglist.itervalues():
self.enable_restoration(stacked)
def get_saved_proxied_obj(self, stacked, request_id):
"""Retrieve the saved object proxied by the specified
StackedObjectProxy for the request identified by request_id"""
# All state for the request identified by request_id
reglist = self.saved_registry_states[request_id][1]
# The top of the stack was current when the exception occurred
stack_level = len(reglist) - 1
stacked_id = id(stacked)
while True:
if stack_level < 0:
# Nothing registered: Call _current_obj_orig to raise a
# TypeError
return stacked._current_obj_orig()
context = reglist[stack_level]
if stacked_id in context:
break
# This StackedObjectProxy may not have been registered by the
# RegistryManager that was active when the exception was raised --
# continue searching down the stack until it's found
stack_level -= 1
return context[stacked_id][1]
def enable_restoration(self, stacked):
"""Replace the specified StackedObjectProxy's methods with their
respective restoration versions.
_current_obj_restoration forces recovery of the saved proxied object
when a restoration context is active in the current thread.
_push/pop_object_restoration avoid pushing/popping data
(pushing/popping is only done at the Registry level) when a restoration
context is active in the current thread"""
if '_current_obj_orig' in stacked.__dict__:
# Restoration already enabled
return
for func_name in ('_current_obj', '_push_object', '_pop_object'):
orig_func = getattr(stacked, func_name)
restoration_func = getattr(stacked, func_name + '_restoration')
stacked.__dict__[func_name + '_orig'] = orig_func
stacked.__dict__[func_name] = restoration_func
def get_request_id(self, environ):
"""Return a unique identifier for the current request"""
from paste.evalexception.middleware import get_debug_count
return get_debug_count(environ)
def restoration_begin(self, request_id):
"""Enable a restoration context in the current thread for the specified
request_id"""
if request_id in self.saved_registry_states:
# Restore the old Registry object's state
registry, reglist = self.saved_registry_states[request_id]
registry.reglist = reglist
self.restoration_context_id.request_id = request_id
def restoration_end(self):
"""Register a restoration context as finished, if one exists"""
try:
del self.restoration_context_id.request_id
except AttributeError:
pass
def in_restoration(self):
"""Determine if a restoration context is active for the current thread.
Returns the request_id it's active for if so, otherwise False"""
return getattr(self.restoration_context_id, 'request_id', False)
restorer = StackedObjectRestorer()
# Paste Deploy entry point
def make_registry_manager(app, global_conf):
return RegistryManager(app)
make_registry_manager.__doc__ = RegistryManager.__doc__

View File

@@ -0,0 +1,178 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
"""
A file monitor and server restarter.
Use this like:
..code-block:: Python
import reloader
reloader.install()
Then make sure your server is installed with a shell script like::
err=3
while test "$err" -eq 3 ; do
python server.py
err="$?"
done
or is run from this .bat file (if you use Windows)::
@echo off
:repeat
python server.py
if %errorlevel% == 3 goto repeat
or run a monitoring process in Python (``paster serve --reload`` does
this).
Use the ``watch_file(filename)`` function to cause a reload/restart for
other other non-Python files (e.g., configuration files). If you have
a dynamic set of files that grows over time you can use something like::
def watch_config_files():
return CONFIG_FILE_CACHE.keys()
paste.reloader.add_file_callback(watch_config_files)
Then every time the reloader polls files it will call
``watch_config_files`` and check all the filenames it returns.
"""
import os
import sys
import time
import threading
import traceback
from paste.util.classinstance import classinstancemethod
def install(poll_interval=1):
"""
Install the reloading monitor.
On some platforms server threads may not terminate when the main
thread does, causing ports to remain open/locked. The
``raise_keyboard_interrupt`` option creates a unignorable signal
which causes the whole application to shut-down (rudely).
"""
mon = Monitor(poll_interval=poll_interval)
t = threading.Thread(target=mon.periodic_reload)
t.setDaemon(True)
t.start()
class Monitor(object):
instances = []
global_extra_files = []
global_file_callbacks = []
def __init__(self, poll_interval):
self.module_mtimes = {}
self.keep_running = True
self.poll_interval = poll_interval
self.extra_files = list(self.global_extra_files)
self.instances.append(self)
self.file_callbacks = list(self.global_file_callbacks)
def periodic_reload(self):
while True:
if not self.check_reload():
# use os._exit() here and not sys.exit() since within a
# thread sys.exit() just closes the given thread and
# won't kill the process; note os._exit does not call
# any atexit callbacks, nor does it do finally blocks,
# flush open files, etc. In otherwords, it is rude.
os._exit(3)
break
time.sleep(self.poll_interval)
def check_reload(self):
filenames = list(self.extra_files)
for file_callback in self.file_callbacks:
try:
filenames.extend(file_callback())
except:
print >> sys.stderr, "Error calling paste.reloader callback %r:" % file_callback
traceback.print_exc()
for module in sys.modules.values():
try:
filename = module.__file__
except (AttributeError, ImportError), exc:
continue
if filename is not None:
filenames.append(filename)
for filename in filenames:
try:
stat = os.stat(filename)
if stat:
mtime = stat.st_mtime
else:
mtime = 0
except (OSError, IOError):
continue
if filename.endswith('.pyc') and os.path.exists(filename[:-1]):
mtime = max(os.stat(filename[:-1]).st_mtime, mtime)
elif filename.endswith('$py.class') and \
os.path.exists(filename[:-9] + '.py'):
mtime = max(os.stat(filename[:-9] + '.py').st_mtime, mtime)
if not self.module_mtimes.has_key(filename):
self.module_mtimes[filename] = mtime
elif self.module_mtimes[filename] < mtime:
print >> sys.stderr, (
"%s changed; reloading..." % filename)
return False
return True
def watch_file(self, cls, filename):
"""Watch the named file for changes"""
filename = os.path.abspath(filename)
if self is None:
for instance in cls.instances:
instance.watch_file(filename)
cls.global_extra_files.append(filename)
else:
self.extra_files.append(filename)
watch_file = classinstancemethod(watch_file)
def add_file_callback(self, cls, callback):
"""Add a callback -- a function that takes no parameters -- that will
return a list of filenames to watch for changes."""
if self is None:
for instance in cls.instances:
instance.add_file_callback(callback)
cls.global_file_callbacks.append(callback)
else:
self.file_callbacks.append(callback)
add_file_callback = classinstancemethod(add_file_callback)
if sys.platform.startswith('java'):
try:
from _systemrestart import SystemRestart
except ImportError:
pass
else:
class JythonMonitor(Monitor):
"""
Monitor that utilizes Jython's special
``_systemrestart.SystemRestart`` exception.
When raised from the main thread it causes Jython to reload
the interpreter in the existing Java process (avoiding
startup time).
Note that this functionality of Jython is experimental and
may change in the future.
"""
def periodic_reload(self):
while True:
if not self.check_reload():
raise SystemRestart()
time.sleep(self.poll_interval)
watch_file = Monitor.watch_file
add_file_callback = Monitor.add_file_callback

View File

@@ -0,0 +1,411 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
# (c) 2005 Ian Bicking and contributors
# This module is part of the Python Paste Project and is released under
# the MIT License: http://www.opensource.org/licenses/mit-license.php
"""
This module provides helper routines with work directly on a WSGI
environment to solve common requirements.
* get_cookies(environ)
* parse_querystring(environ)
* parse_formvars(environ, include_get_vars=True)
* construct_url(environ, with_query_string=True, with_path_info=True,
script_name=None, path_info=None, querystring=None)
* path_info_split(path_info)
* path_info_pop(environ)
* resolve_relative_url(url, environ)
"""
import cgi
from Cookie import SimpleCookie, CookieError
from StringIO import StringIO
import urlparse
import urllib
try:
from UserDict import DictMixin
except ImportError:
from paste.util.UserDict24 import DictMixin
from paste.util.multidict import MultiDict
__all__ = ['get_cookies', 'get_cookie_dict', 'parse_querystring',
'parse_formvars', 'construct_url', 'path_info_split',
'path_info_pop', 'resolve_relative_url', 'EnvironHeaders']
def get_cookies(environ):
"""
Gets a cookie object (which is a dictionary-like object) from the
request environment; caches this value in case get_cookies is
called again for the same request.
"""
header = environ.get('HTTP_COOKIE', '')
if environ.has_key('paste.cookies'):
cookies, check_header = environ['paste.cookies']
if check_header == header:
return cookies
cookies = SimpleCookie()
try:
cookies.load(header)
except CookieError:
pass
environ['paste.cookies'] = (cookies, header)
return cookies
def get_cookie_dict(environ):
"""Return a *plain* dictionary of cookies as found in the request.
Unlike ``get_cookies`` this returns a dictionary, not a
``SimpleCookie`` object. For incoming cookies a dictionary fully
represents the information. Like ``get_cookies`` this caches and
checks the cache.
"""
header = environ.get('HTTP_COOKIE')
if not header:
return {}
if environ.has_key('paste.cookies.dict'):
cookies, check_header = environ['paste.cookies.dict']
if check_header == header:
return cookies
cookies = SimpleCookie()
try:
cookies.load(header)
except CookieError:
pass
result = {}
for name in cookies:
result[name] = cookies[name].value
environ['paste.cookies.dict'] = (result, header)
return result
def parse_querystring(environ):
"""
Parses a query string into a list like ``[(name, value)]``.
Caches this value in case parse_querystring is called again
for the same request.
You can pass the result to ``dict()``, but be aware that keys that
appear multiple times will be lost (only the last value will be
preserved).
"""
source = environ.get('QUERY_STRING', '')
if not source:
return []
if 'paste.parsed_querystring' in environ:
parsed, check_source = environ['paste.parsed_querystring']
if check_source == source:
return parsed
parsed = cgi.parse_qsl(source, keep_blank_values=True,
strict_parsing=False)
environ['paste.parsed_querystring'] = (parsed, source)
return parsed
def parse_dict_querystring(environ):
"""Parses a query string like parse_querystring, but returns a MultiDict
Caches this value in case parse_dict_querystring is called again
for the same request.
Example::
>>> environ = {'QUERY_STRING': 'day=Monday&user=fred&user=jane'}
>>> parsed = parse_dict_querystring(environ)
>>> parsed['day']
'Monday'
>>> parsed['user']
'fred'
>>> parsed.getall('user')
['fred', 'jane']
"""
source = environ.get('QUERY_STRING', '')
if not source:
return MultiDict()
if 'paste.parsed_dict_querystring' in environ:
parsed, check_source = environ['paste.parsed_dict_querystring']
if check_source == source:
return parsed
parsed = cgi.parse_qsl(source, keep_blank_values=True,
strict_parsing=False)
multi = MultiDict(parsed)
environ['paste.parsed_dict_querystring'] = (multi, source)
return multi
def parse_formvars(environ, include_get_vars=True):
"""Parses the request, returning a MultiDict of form variables.
If ``include_get_vars`` is true then GET (query string) variables
will also be folded into the MultiDict.
All values should be strings, except for file uploads which are
left as ``FieldStorage`` instances.
If the request was not a normal form request (e.g., a POST with an
XML body) then ``environ['wsgi.input']`` won't be read.
"""
source = environ['wsgi.input']
if 'paste.parsed_formvars' in environ:
parsed, check_source = environ['paste.parsed_formvars']
if check_source == source:
if include_get_vars:
parsed.update(parse_querystring(environ))
return parsed
# @@: Shouldn't bother FieldStorage parsing during GET/HEAD and
# fake_out_cgi requests
type = environ.get('CONTENT_TYPE', '').lower()
if ';' in type:
type = type.split(';', 1)[0]
fake_out_cgi = type not in ('', 'application/x-www-form-urlencoded',
'multipart/form-data')
# FieldStorage assumes a default CONTENT_LENGTH of -1, but a
# default of 0 is better:
if not environ.get('CONTENT_LENGTH'):
environ['CONTENT_LENGTH'] = '0'
# Prevent FieldStorage from parsing QUERY_STRING during GET/HEAD
# requests
old_query_string = environ.get('QUERY_STRING','')
environ['QUERY_STRING'] = ''
if fake_out_cgi:
input = StringIO('')
old_content_type = environ.get('CONTENT_TYPE')
old_content_length = environ.get('CONTENT_LENGTH')
environ['CONTENT_LENGTH'] = '0'
environ['CONTENT_TYPE'] = ''
else:
input = environ['wsgi.input']
fs = cgi.FieldStorage(fp=input,
environ=environ,
keep_blank_values=1)
environ['QUERY_STRING'] = old_query_string
if fake_out_cgi:
environ['CONTENT_TYPE'] = old_content_type
environ['CONTENT_LENGTH'] = old_content_length
formvars = MultiDict()
if isinstance(fs.value, list):
for name in fs.keys():
values = fs[name]
if not isinstance(values, list):
values = [values]
for value in values:
if not value.filename:
value = value.value
formvars.add(name, value)
environ['paste.parsed_formvars'] = (formvars, source)
if include_get_vars:
formvars.update(parse_querystring(environ))
return formvars
def construct_url(environ, with_query_string=True, with_path_info=True,
script_name=None, path_info=None, querystring=None):
"""Reconstructs the URL from the WSGI environment.
You may override SCRIPT_NAME, PATH_INFO, and QUERYSTRING with
the keyword arguments.
"""
url = environ['wsgi.url_scheme']+'://'
if environ.get('HTTP_HOST'):
host = environ['HTTP_HOST']
port = None
if ':' in host:
host, port = host.split(':', 1)
if environ['wsgi.url_scheme'] == 'https':
if port == '443':
port = None
elif environ['wsgi.url_scheme'] == 'http':
if port == '80':
port = None
url += host
if port:
url += ':%s' % port
else:
url += environ['SERVER_NAME']
if environ['wsgi.url_scheme'] == 'https':
if environ['SERVER_PORT'] != '443':
url += ':' + environ['SERVER_PORT']
else:
if environ['SERVER_PORT'] != '80':
url += ':' + environ['SERVER_PORT']
if script_name is None:
url += urllib.quote(environ.get('SCRIPT_NAME',''))
else:
url += urllib.quote(script_name)
if with_path_info:
if path_info is None:
url += urllib.quote(environ.get('PATH_INFO',''))
else:
url += urllib.quote(path_info)
if with_query_string:
if querystring is None:
if environ.get('QUERY_STRING'):
url += '?' + environ['QUERY_STRING']
elif querystring:
url += '?' + querystring
return url
def resolve_relative_url(url, environ):
"""
Resolve the given relative URL as being relative to the
location represented by the environment. This can be used
for redirecting to a relative path. Note: if url is already
absolute, this function will (intentionally) have no effect
on it.
"""
cur_url = construct_url(environ, with_query_string=False)
return urlparse.urljoin(cur_url, url)
def path_info_split(path_info):
"""
Splits off the first segment of the path. Returns (first_part,
rest_of_path). first_part can be None (if PATH_INFO is empty), ''
(if PATH_INFO is '/'), or a name without any /'s. rest_of_path
can be '' or a string starting with /.
"""
if not path_info:
return None, ''
assert path_info.startswith('/'), (
"PATH_INFO should start with /: %r" % path_info)
path_info = path_info.lstrip('/')
if '/' in path_info:
first, rest = path_info.split('/', 1)
return first, '/' + rest
else:
return path_info, ''
def path_info_pop(environ):
"""
'Pops' off the next segment of PATH_INFO, pushing it onto
SCRIPT_NAME, and returning that segment.
For instance::
>>> def call_it(script_name, path_info):
... env = {'SCRIPT_NAME': script_name, 'PATH_INFO': path_info}
... result = path_info_pop(env)
... print 'SCRIPT_NAME=%r; PATH_INFO=%r; returns=%r' % (
... env['SCRIPT_NAME'], env['PATH_INFO'], result)
>>> call_it('/foo', '/bar')
SCRIPT_NAME='/foo/bar'; PATH_INFO=''; returns='bar'
>>> call_it('/foo/bar', '')
SCRIPT_NAME='/foo/bar'; PATH_INFO=''; returns=None
>>> call_it('/foo/bar', '/')
SCRIPT_NAME='/foo/bar/'; PATH_INFO=''; returns=''
>>> call_it('', '/1/2/3')
SCRIPT_NAME='/1'; PATH_INFO='/2/3'; returns='1'
>>> call_it('', '//1/2')
SCRIPT_NAME='//1'; PATH_INFO='/2'; returns='1'
"""
path = environ.get('PATH_INFO', '')
if not path:
return None
while path.startswith('/'):
environ['SCRIPT_NAME'] += '/'
path = path[1:]
if '/' not in path:
environ['SCRIPT_NAME'] += path
environ['PATH_INFO'] = ''
return path
else:
segment, path = path.split('/', 1)
environ['PATH_INFO'] = '/' + path
environ['SCRIPT_NAME'] += segment
return segment
_parse_headers_special = {
# This is a Zope convention, but we'll allow it here:
'HTTP_CGI_AUTHORIZATION': 'Authorization',
'CONTENT_LENGTH': 'Content-Length',
'CONTENT_TYPE': 'Content-Type',
}
def parse_headers(environ):
"""
Parse the headers in the environment (like ``HTTP_HOST``) and
yield a sequence of those (header_name, value) tuples.
"""
# @@: Maybe should parse out comma-separated headers?
for cgi_var, value in environ.iteritems():
if cgi_var in _parse_headers_special:
yield _parse_headers_special[cgi_var], value
elif cgi_var.startswith('HTTP_'):
yield cgi_var[5:].title().replace('_', '-'), value
class EnvironHeaders(DictMixin):
"""An object that represents the headers as present in a
WSGI environment.
This object is a wrapper (with no internal state) for a WSGI
request object, representing the CGI-style HTTP_* keys as a
dictionary. Because a CGI environment can only hold one value for
each key, this dictionary is single-valued (unlike outgoing
headers).
"""
def __init__(self, environ):
self.environ = environ
def _trans_name(self, name):
key = 'HTTP_'+name.replace('-', '_').upper()
if key == 'HTTP_CONTENT_LENGTH':
key = 'CONTENT_LENGTH'
elif key == 'HTTP_CONTENT_TYPE':
key = 'CONTENT_TYPE'
return key
def _trans_key(self, key):
if key == 'CONTENT_TYPE':
return 'Content-Type'
elif key == 'CONTENT_LENGTH':
return 'Content-Length'
elif key.startswith('HTTP_'):
return key[5:].replace('_', '-').title()
else:
return None
def __getitem__(self, item):
return self.environ[self._trans_name(item)]
def __setitem__(self, item, value):
# @@: Should this dictionary be writable at all?
self.environ[self._trans_name(item)] = value
def __delitem__(self, item):
del self.environ[self._trans_name(item)]
def __iter__(self):
for key in self.environ:
name = self._trans_key(key)
if name is not None:
yield name
def keys(self):
return list(iter(self))
def __contains__(self, item):
return self._trans_name(item) in self.environ
def _cgi_FieldStorage__repr__patch(self):
""" monkey patch for FieldStorage.__repr__
Unbelievely, the default __repr__ on FieldStorage reads
the entire file content instead of being sane about it.
This is a simple replacement that doesn't do that
"""
if self.file:
return "FieldStorage(%r, %r)" % (
self.name, self.filename)
return "FieldStorage(%r, %r, %r)" % (
self.name, self.filename, self.value)
cgi.FieldStorage.__repr__ = _cgi_FieldStorage__repr__patch
if __name__ == '__main__':
import doctest
doctest.testmod()

View File

@@ -0,0 +1,240 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
"""Routines to generate WSGI responses"""
############################################################
## Headers
############################################################
import warnings
class HeaderDict(dict):
"""
This represents response headers. It handles the headers as a
dictionary, with case-insensitive keys.
Also there is an ``.add(key, value)`` method, which sets the key,
or adds the value to the current value (turning it into a list if
necessary).
For passing to WSGI there is a ``.headeritems()`` method which is
like ``.items()`` but unpacks value that are lists. It also
handles encoding -- all headers are encoded in ASCII (if they are
unicode).
@@: Should that encoding be ISO-8859-1 or UTF-8? I'm not sure
what the spec says.
"""
def __getitem__(self, key):
return dict.__getitem__(self, self.normalize(key))
def __setitem__(self, key, value):
dict.__setitem__(self, self.normalize(key), value)
def __delitem__(self, key):
dict.__delitem__(self, self.normalize(key))
def __contains__(self, key):
return dict.__contains__(self, self.normalize(key))
has_key = __contains__
def get(self, key, failobj=None):
return dict.get(self, self.normalize(key), failobj)
def setdefault(self, key, failobj=None):
return dict.setdefault(self, self.normalize(key), failobj)
def pop(self, key, *args):
return dict.pop(self, self.normalize(key), *args)
def update(self, other):
for key in other:
self[self.normalize(key)] = other[key]
def normalize(self, key):
return str(key).lower().strip()
def add(self, key, value):
key = self.normalize(key)
if key in self:
if isinstance(self[key], list):
self[key].append(value)
else:
self[key] = [self[key], value]
else:
self[key] = value
def headeritems(self):
result = []
for key, value in self.items():
if isinstance(value, list):
for v in value:
result.append((key, str(v)))
else:
result.append((key, str(value)))
return result
#@classmethod
def fromlist(cls, seq):
self = cls()
for name, value in seq:
self.add(name, value)
return self
fromlist = classmethod(fromlist)
def has_header(headers, name):
"""
Is header named ``name`` present in headers?
"""
name = name.lower()
for header, value in headers:
if header.lower() == name:
return True
return False
def header_value(headers, name):
"""
Returns the header's value, or None if no such header. If a
header appears more than once, all the values of the headers
are joined with ','. Note that this is consistent /w RFC 2616
section 4.2 which states:
It MUST be possible to combine the multiple header fields
into one "field-name: field-value" pair, without changing
the semantics of the message, by appending each subsequent
field-value to the first, each separated by a comma.
However, note that the original netscape usage of 'Set-Cookie',
especially in MSIE which contains an 'expires' date will is not
compatible with this particular concatination method.
"""
name = name.lower()
result = [value for header, value in headers
if header.lower() == name]
if result:
return ','.join(result)
else:
return None
def remove_header(headers, name):
"""
Removes the named header from the list of headers. Returns the
value of that header, or None if no header found. If multiple
headers are found, only the last one is returned.
"""
name = name.lower()
i = 0
result = None
while i < len(headers):
if headers[i][0].lower() == name:
result = headers[i][1]
del headers[i]
continue
i += 1
return result
def replace_header(headers, name, value):
"""
Updates the headers replacing the first occurance of the given name
with the value provided; asserting that no further occurances
happen. Note that this is _not_ the same as remove_header and then
append, as two distinct operations (del followed by an append) are
not atomic in a threaded environment. Returns the previous header
value for the provided name, if any. Clearly one should not use
this function with ``set-cookie`` or other names that may have more
than one occurance in the headers.
"""
name = name.lower()
i = 0
result = None
while i < len(headers):
if headers[i][0].lower() == name:
assert not result, "two values for the header '%s' found" % name
result = headers[i][1]
headers[i] = (name, value)
i += 1
if not result:
headers.append((name, value))
return result
############################################################
## Deprecated methods
############################################################
def error_body_response(error_code, message, __warn=True):
"""
Returns a standard HTML response page for an HTTP error.
**Note:** Deprecated
"""
if __warn:
warnings.warn(
'wsgilib.error_body_response is deprecated; use the '
'wsgi_application method on an HTTPException object '
'instead', DeprecationWarning, 2)
return '''\
<html>
<head>
<title>%(error_code)s</title>
</head>
<body>
<h1>%(error_code)s</h1>
%(message)s
</body>
</html>''' % {
'error_code': error_code,
'message': message,
}
def error_response(environ, error_code, message,
debug_message=None, __warn=True):
"""
Returns the status, headers, and body of an error response.
Use like:
.. code-block:: python
status, headers, body = wsgilib.error_response(
'301 Moved Permanently', 'Moved to <a href="%s">%s</a>'
% (url, url))
start_response(status, headers)
return [body]
**Note:** Deprecated
"""
if __warn:
warnings.warn(
'wsgilib.error_response is deprecated; use the '
'wsgi_application method on an HTTPException object '
'instead', DeprecationWarning, 2)
if debug_message and environ.get('paste.config', {}).get('debug'):
message += '\n\n<!-- %s -->' % debug_message
body = error_body_response(error_code, message, __warn=False)
headers = [('content-type', 'text/html'),
('content-length', str(len(body)))]
return error_code, headers, body
def error_response_app(error_code, message, debug_message=None,
__warn=True):
"""
An application that emits the given error response.
**Note:** Deprecated
"""
if __warn:
warnings.warn(
'wsgilib.error_response_app is deprecated; use the '
'wsgi_application method on an HTTPException object '
'instead', DeprecationWarning, 2)
def application(environ, start_response):
status, headers, body = error_response(
environ, error_code, message,
debug_message=debug_message, __warn=False)
start_response(status, headers)
return [body]
return application

View File

@@ -0,0 +1,337 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
"""
Creates a session object in your WSGI environment.
Use like:
..code-block:: Python
environ['paste.session.factory']()
This will return a dictionary. The contents of this dictionary will
be saved to disk when the request is completed. The session will be
created when you first fetch the session dictionary, and a cookie will
be sent in that case. There's current no way to use sessions without
cookies, and there's no way to delete a session except to clear its
data.
@@: This doesn't do any locking, and may cause problems when a single
session is accessed concurrently. Also, it loads and saves the
session for each request, with no caching. Also, sessions aren't
expired.
"""
from Cookie import SimpleCookie
import time
import random
import os
import datetime
import threading
import tempfile
try:
import cPickle
except ImportError:
import pickle as cPickle
try:
from hashlib import md5
except ImportError:
from md5 import md5
from paste import wsgilib
from paste import request
class SessionMiddleware(object):
def __init__(self, application, global_conf=None, **factory_kw):
self.application = application
self.factory_kw = factory_kw
def __call__(self, environ, start_response):
session_factory = SessionFactory(environ, **self.factory_kw)
environ['paste.session.factory'] = session_factory
remember_headers = []
def session_start_response(status, headers, exc_info=None):
if not session_factory.created:
remember_headers[:] = [status, headers]
return start_response(status, headers)
headers.append(session_factory.set_cookie_header())
return start_response(status, headers, exc_info)
app_iter = self.application(environ, session_start_response)
def start():
if session_factory.created and remember_headers:
# Tricky bastard used the session after start_response
status, headers = remember_headers
headers.append(session_factory.set_cookie_header())
exc = ValueError(
"You cannot get the session after content from the "
"app_iter has been returned")
start_response(status, headers, (exc.__class__, exc, None))
def close():
if session_factory.used:
session_factory.close()
return wsgilib.add_start_close(app_iter, start, close)
class SessionFactory(object):
def __init__(self, environ, cookie_name='_SID_',
session_class=None,
session_expiration=60*12, # in minutes
**session_class_kw):
self.created = False
self.used = False
self.environ = environ
self.cookie_name = cookie_name
self.session = None
self.session_class = session_class or FileSession
self.session_class_kw = session_class_kw
self.expiration = session_expiration
def __call__(self):
self.used = True
if self.session is not None:
return self.session.data()
cookies = request.get_cookies(self.environ)
session = None
if cookies.has_key(self.cookie_name):
self.sid = cookies[self.cookie_name].value
try:
session = self.session_class(self.sid, create=False,
**self.session_class_kw)
except KeyError:
# Invalid SID
pass
if session is None:
self.created = True
self.sid = self.make_sid()
session = self.session_class(self.sid, create=True,
**self.session_class_kw)
session.clean_up()
self.session = session
return session.data()
def has_session(self):
if self.session is not None:
return True
cookies = request.get_cookies(self.environ)
if cookies.has_key(self.cookie_name):
return True
return False
def make_sid(self):
# @@: need better algorithm
return (''.join(['%02d' % x for x in time.localtime(time.time())[:6]])
+ '-' + self.unique_id())
def unique_id(self, for_object=None):
"""
Generates an opaque, identifier string that is practically
guaranteed to be unique. If an object is passed, then its
id() is incorporated into the generation. Relies on md5 and
returns a 32 character long string.
"""
r = [time.time(), random.random()]
if hasattr(os, 'times'):
r.append(os.times())
if for_object is not None:
r.append(id(for_object))
md5_hash = md5(str(r))
try:
return md5_hash.hexdigest()
except AttributeError:
# Older versions of Python didn't have hexdigest, so we'll
# do it manually
hexdigest = []
for char in md5_hash.digest():
hexdigest.append('%02x' % ord(char))
return ''.join(hexdigest)
def set_cookie_header(self):
c = SimpleCookie()
c[self.cookie_name] = self.sid
c[self.cookie_name]['path'] = '/'
gmt_expiration_time = time.gmtime(time.time() + (self.expiration * 60))
c[self.cookie_name]['expires'] = time.strftime("%a, %d-%b-%Y %H:%M:%S GMT", gmt_expiration_time)
name, value = str(c).split(': ', 1)
return (name, value)
def close(self):
if self.session is not None:
self.session.close()
last_cleanup = None
cleaning_up = False
cleanup_cycle = datetime.timedelta(seconds=15*60) #15 min
class FileSession(object):
def __init__(self, sid, create=False, session_file_path=tempfile.gettempdir(),
chmod=None,
expiration=2880, # in minutes: 48 hours
):
if chmod and isinstance(chmod, basestring):
chmod = int(chmod, 8)
self.chmod = chmod
if not sid:
# Invalid...
raise KeyError
self.session_file_path = session_file_path
self.sid = sid
if not create:
if not os.path.exists(self.filename()):
raise KeyError
self._data = None
self.expiration = expiration
def filename(self):
return os.path.join(self.session_file_path, self.sid)
def data(self):
if self._data is not None:
return self._data
if os.path.exists(self.filename()):
f = open(self.filename(), 'rb')
self._data = cPickle.load(f)
f.close()
else:
self._data = {}
return self._data
def close(self):
if self._data is not None:
filename = self.filename()
exists = os.path.exists(filename)
if not self._data:
if exists:
os.unlink(filename)
else:
f = open(filename, 'wb')
cPickle.dump(self._data, f)
f.close()
if not exists and self.chmod:
os.chmod(filename, self.chmod)
def _clean_up(self):
global cleaning_up
try:
exp_time = datetime.timedelta(seconds=self.expiration*60)
now = datetime.datetime.now()
#Open every session and check that it isn't too old
for root, dirs, files in os.walk(self.session_file_path):
for f in files:
self._clean_up_file(f, exp_time=exp_time, now=now)
finally:
cleaning_up = False
def _clean_up_file(self, f, exp_time, now):
t = f.split("-")
if len(t) != 2:
return
t = t[0]
try:
sess_time = datetime.datetime(
int(t[0:4]),
int(t[4:6]),
int(t[6:8]),
int(t[8:10]),
int(t[10:12]),
int(t[12:14]))
except ValueError:
# Probably not a session file at all
return
if sess_time + exp_time < now:
os.remove(os.path.join(self.session_file_path, f))
def clean_up(self):
global last_cleanup, cleanup_cycle, cleaning_up
now = datetime.datetime.now()
if cleaning_up:
return
if not last_cleanup or last_cleanup + cleanup_cycle < now:
if not cleaning_up:
cleaning_up = True
try:
last_cleanup = now
t = threading.Thread(target=self._clean_up)
t.start()
except:
# Normally _clean_up should set cleaning_up
# to false, but if something goes wrong starting
# it...
cleaning_up = False
raise
class _NoDefault(object):
def __repr__(self):
return '<dynamic default>'
NoDefault = _NoDefault()
def make_session_middleware(
app, global_conf,
session_expiration=NoDefault,
expiration=NoDefault,
cookie_name=NoDefault,
session_file_path=NoDefault,
chmod=NoDefault):
"""
Adds a middleware that handles sessions for your applications.
The session is a peristent dictionary. To get this dictionary
in your application, use ``environ['paste.session.factory']()``
which returns this persistent dictionary.
Configuration:
session_expiration:
The time each session lives, in minutes. This controls
the cookie expiration. Default 12 hours.
expiration:
The time each session lives on disk. Old sessions are
culled from disk based on this. Default 48 hours.
cookie_name:
The cookie name used to track the session. Use different
names to avoid session clashes.
session_file_path:
Sessions are put in this location, default /tmp.
chmod:
The octal chmod you want to apply to new sessions (e.g., 660
to make the sessions group readable/writable)
Each of these also takes from the global configuration. cookie_name
and chmod take from session_cookie_name and session_chmod
"""
if session_expiration is NoDefault:
session_expiration = global_conf.get('session_expiration', 60*12)
session_expiration = int(session_expiration)
if expiration is NoDefault:
expiration = global_conf.get('expiration', 60*48)
expiration = int(expiration)
if cookie_name is NoDefault:
cookie_name = global_conf.get('session_cookie_name', '_SID_')
if session_file_path is NoDefault:
session_file_path = global_conf.get('session_file_path', '/tmp')
if chmod is NoDefault:
chmod = global_conf.get('session_chmod', None)
return SessionMiddleware(
app, session_expiration=session_expiration,
expiration=expiration, cookie_name=cookie_name,
session_file_path=session_file_path, chmod=chmod)

View File

@@ -0,0 +1,120 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
# (c) 2005 Clark C. Evans
# This module is part of the Python Paste Project and is released under
# the MIT License: http://www.opensource.org/licenses/mit-license.php
"""
Middleware related to transactions and database connections.
At this time it is very basic; but will eventually sprout all that
two-phase commit goodness that I don't need.
.. note::
This is experimental, and will change in the future.
"""
from paste.httpexceptions import HTTPException
from wsgilib import catch_errors
class TransactionManagerMiddleware(object):
def __init__(self, application):
self.application = application
def __call__(self, environ, start_response):
environ['paste.transaction_manager'] = manager = Manager()
# This makes sure nothing else traps unexpected exceptions:
environ['paste.throw_errors'] = True
return catch_errors(self.application, environ, start_response,
error_callback=manager.error,
ok_callback=manager.finish)
class Manager(object):
def __init__(self):
self.aborted = False
self.transactions = []
def abort(self):
self.aborted = True
def error(self, exc_info):
self.aborted = True
self.finish()
def finish(self):
for trans in self.transactions:
if self.aborted:
trans.rollback()
else:
trans.commit()
class ConnectionFactory(object):
"""
Provides a callable interface for connecting to ADBAPI databases in
a WSGI style (using the environment). More advanced connection
factories might use the REMOTE_USER and/or other environment
variables to make the connection returned depend upon the request.
"""
def __init__(self, module, *args, **kwargs):
#assert getattr(module,'threadsaftey',0) > 0
self.module = module
self.args = args
self.kwargs = kwargs
# deal with database string quoting issues
self.quote = lambda s: "'%s'" % s.replace("'","''")
if hasattr(self.module,'PgQuoteString'):
self.quote = self.module.PgQuoteString
def __call__(self, environ=None):
conn = self.module.connect(*self.args, **self.kwargs)
conn.__dict__['module'] = self.module
conn.__dict__['quote'] = self.quote
return conn
def BasicTransactionHandler(application, factory):
"""
Provides a simple mechanism for starting a transaction based on the
factory; and for either committing or rolling back the transaction
depending on the result. It checks for the response's current
status code either through the latest call to start_response; or
through a HTTPException's code. If it is a 100, 200, or 300; the
transaction is committed; otherwise it is rolled back.
"""
def basic_transaction(environ, start_response):
conn = factory(environ)
environ['paste.connection'] = conn
should_commit = [500]
def finalizer(exc_info=None):
if exc_info:
if isinstance(exc_info[1], HTTPException):
should_commit.append(exc_info[1].code)
if should_commit.pop() < 400:
conn.commit()
else:
try:
conn.rollback()
except:
# TODO: check if rollback has already happened
return
conn.close()
def basictrans_start_response(status, headers, exc_info = None):
should_commit.append(int(status.split(" ")[0]))
return start_response(status, headers, exc_info)
return catch_errors(application, environ, basictrans_start_response,
finalizer, finalizer)
return basic_transaction
__all__ = ['ConnectionFactory', 'BasicTransactionHandler']
if '__main__' == __name__ and False:
from pyPgSQL import PgSQL
factory = ConnectionFactory(PgSQL, database="testing")
conn = factory()
curr = conn.cursor()
curr.execute("SELECT now(), %s" % conn.quote("B'n\\'gles"))
(time, bing) = curr.fetchone()
print bing, time

View File

@@ -0,0 +1,121 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
"""
Middleware for logging requests, using Apache combined log format
"""
import logging
import time
import urllib
class TransLogger(object):
"""
This logging middleware will log all requests as they go through.
They are, by default, sent to a logger named ``'wsgi'`` at the
INFO level.
If ``setup_console_handler`` is true, then messages for the named
logger will be sent to the console.
"""
format = ('%(REMOTE_ADDR)s - %(REMOTE_USER)s [%(time)s] '
'"%(REQUEST_METHOD)s %(REQUEST_URI)s %(HTTP_VERSION)s" '
'%(status)s %(bytes)s "%(HTTP_REFERER)s" "%(HTTP_USER_AGENT)s"')
def __init__(self, application,
logger=None,
format=None,
logging_level=logging.INFO,
logger_name='wsgi',
setup_console_handler=True,
set_logger_level=logging.DEBUG):
if format is not None:
self.format = format
self.application = application
self.logging_level = logging_level
self.logger_name = logger_name
if logger is None:
self.logger = logging.getLogger(self.logger_name)
if setup_console_handler:
console = logging.StreamHandler()
console.setLevel(logging.DEBUG)
# We need to control the exact format:
console.setFormatter(logging.Formatter('%(message)s'))
self.logger.addHandler(console)
self.logger.propagate = False
if set_logger_level is not None:
self.logger.setLevel(set_logger_level)
else:
self.logger = logger
def __call__(self, environ, start_response):
start = time.localtime()
req_uri = urllib.quote(environ.get('SCRIPT_NAME', '')
+ environ.get('PATH_INFO', ''))
if environ.get('QUERY_STRING'):
req_uri += '?'+environ['QUERY_STRING']
method = environ['REQUEST_METHOD']
def replacement_start_response(status, headers, exc_info=None):
# @@: Ideally we would count the bytes going by if no
# content-length header was provided; but that does add
# some overhead, so at least for now we'll be lazy.
bytes = None
for name, value in headers:
if name.lower() == 'content-length':
bytes = value
self.write_log(environ, method, req_uri, start, status, bytes)
return start_response(status, headers)
return self.application(environ, replacement_start_response)
def write_log(self, environ, method, req_uri, start, status, bytes):
if bytes is None:
bytes = '-'
if time.daylight:
offset = time.altzone / 60 / 60 * -100
else:
offset = time.timezone / 60 / 60 * -100
if offset >= 0:
offset = "+%0.4d" % (offset)
elif offset < 0:
offset = "%0.4d" % (offset)
remote_addr = '-'
if environ.get('HTTP_X_FORWARDED_FOR'):
remote_addr = environ['HTTP_X_FORWARDED_FOR']
elif environ.get('REMOTE_ADDR'):
remote_addr = environ['REMOTE_ADDR']
d = {
'REMOTE_ADDR': remote_addr,
'REMOTE_USER': environ.get('REMOTE_USER') or '-',
'REQUEST_METHOD': method,
'REQUEST_URI': req_uri,
'HTTP_VERSION': environ.get('SERVER_PROTOCOL'),
'time': time.strftime('%d/%b/%Y:%H:%M:%S ', start) + offset,
'status': status.split(None, 1)[0],
'bytes': bytes,
'HTTP_REFERER': environ.get('HTTP_REFERER', '-'),
'HTTP_USER_AGENT': environ.get('HTTP_USER_AGENT', '-'),
}
message = self.format % d
self.logger.log(self.logging_level, message)
def make_filter(
app, global_conf,
logger_name='wsgi',
format=None,
logging_level=logging.INFO,
setup_console_handler=True,
set_logger_level=logging.DEBUG):
from paste.util.converters import asbool
if isinstance(logging_level, basestring):
logging_level = logging._levelNames[logging_level]
if isinstance(set_logger_level, basestring):
set_logger_level = logging._levelNames[set_logger_level]
return TransLogger(
app,
format=format or None,
logging_level=logging_level,
logger_name=logger_name,
setup_console_handler=asbool(setup_console_handler),
set_logger_level=set_logger_level)
make_filter.__doc__ = TransLogger.__doc__

View File

@@ -0,0 +1,475 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
"""
This module implements a class for handling URLs.
"""
import urllib
import cgi
from paste import request
# Imported lazily from FormEncode:
variabledecode = None
__all__ = ["URL", "Image"]
def html_quote(v):
if v is None:
return ''
return cgi.escape(str(v), 1)
def url_quote(v):
if v is None:
return ''
return urllib.quote(str(v))
url_unquote = urllib.unquote
def js_repr(v):
if v is None:
return 'null'
elif v is False:
return 'false'
elif v is True:
return 'true'
elif isinstance(v, list):
return '[%s]' % ', '.join(map(js_repr, v))
elif isinstance(v, dict):
return '{%s}' % ', '.join(
['%s: %s' % (js_repr(key), js_repr(value))
for key, value in v])
elif isinstance(v, str):
return repr(v)
elif isinstance(v, unicode):
# @@: how do you do Unicode literals in Javascript?
return repr(v.encode('UTF-8'))
elif isinstance(v, (float, int)):
return repr(v)
elif isinstance(v, long):
return repr(v).lstrip('L')
elif hasattr(v, '__js_repr__'):
return v.__js_repr__()
else:
raise ValueError(
"I don't know how to turn %r into a Javascript representation"
% v)
class URLResource(object):
"""
This is an abstract superclass for different kinds of URLs
"""
default_params = {}
def __init__(self, url, vars=None, attrs=None,
params=None):
self.url = url or '/'
self.vars = vars or []
self.attrs = attrs or {}
self.params = self.default_params.copy()
self.original_params = params or {}
if params:
self.params.update(params)
#@classmethod
def from_environ(cls, environ, with_query_string=True,
with_path_info=True, script_name=None,
path_info=None, querystring=None):
url = request.construct_url(
environ, with_query_string=False,
with_path_info=with_path_info, script_name=script_name,
path_info=path_info)
if with_query_string:
if querystring is None:
vars = request.parse_querystring(environ)
else:
vars = cgi.parse_qsl(
querystring,
keep_blank_values=True,
strict_parsing=False)
else:
vars = None
v = cls(url, vars=vars)
return v
from_environ = classmethod(from_environ)
def __call__(self, *args, **kw):
res = self._add_positional(args)
res = res._add_vars(kw)
return res
def __getitem__(self, item):
if '=' in item:
name, value = item.split('=', 1)
return self._add_vars({url_unquote(name): url_unquote(value)})
return self._add_positional((item,))
def attr(self, **kw):
for key in kw.keys():
if key.endswith('_'):
kw[key[:-1]] = kw[key]
del kw[key]
new_attrs = self.attrs.copy()
new_attrs.update(kw)
return self.__class__(self.url, vars=self.vars,
attrs=new_attrs,
params=self.original_params)
def param(self, **kw):
new_params = self.original_params.copy()
new_params.update(kw)
return self.__class__(self.url, vars=self.vars,
attrs=self.attrs,
params=new_params)
def coerce_vars(self, vars):
global variabledecode
need_variable_encode = False
for key, value in vars.items():
if isinstance(value, dict):
need_variable_encode = True
if key.endswith('_'):
vars[key[:-1]] = vars[key]
del vars[key]
if need_variable_encode:
if variabledecode is None:
from formencode import variabledecode
vars = variabledecode.variable_encode(vars)
return vars
def var(self, **kw):
kw = self.coerce_vars(kw)
new_vars = self.vars + kw.items()
return self.__class__(self.url, vars=new_vars,
attrs=self.attrs,
params=self.original_params)
def setvar(self, **kw):
"""
Like ``.var(...)``, except overwrites keys, where .var simply
extends the keys. Setting a variable to None here will
effectively delete it.
"""
kw = self.coerce_vars(kw)
new_vars = []
for name, values in self.vars:
if name in kw:
continue
new_vars.append((name, values))
new_vars.extend(kw.items())
return self.__class__(self.url, vars=new_vars,
attrs=self.attrs,
params=self.original_params)
def setvars(self, **kw):
"""
Creates a copy of this URL, but with all the variables set/reset
(like .setvar(), except clears past variables at the same time)
"""
return self.__class__(self.url, vars=kw.items(),
attrs=self.attrs,
params=self.original_params)
def addpath(self, *paths):
u = self
for path in paths:
path = str(path).lstrip('/')
new_url = u.url
if not new_url.endswith('/'):
new_url += '/'
u = u.__class__(new_url+path, vars=u.vars,
attrs=u.attrs,
params=u.original_params)
return u
__div__ = addpath
def become(self, OtherClass):
return OtherClass(self.url, vars=self.vars,
attrs=self.attrs,
params=self.original_params)
def href__get(self):
s = self.url
if self.vars:
s += '?'
vars = []
for name, val in self.vars:
if isinstance(val, (list, tuple)):
val = [v for v in val if v is not None]
elif val is None:
continue
vars.append((name, val))
s += urllib.urlencode(vars, True)
return s
href = property(href__get)
def __repr__(self):
base = '<%s %s' % (self.__class__.__name__,
self.href or "''")
if self.attrs:
base += ' attrs(%s)' % (
' '.join(['%s="%s"' % (html_quote(n), html_quote(v))
for n, v in self.attrs.items()]))
if self.original_params:
base += ' params(%s)' % (
', '.join(['%s=%r' % (n, v)
for n, v in self.attrs.items()]))
return base + '>'
def html__get(self):
if not self.params.get('tag'):
raise ValueError(
"You cannot get the HTML of %r until you set the "
"'tag' param'" % self)
content = self._get_content()
tag = '<%s' % self.params.get('tag')
attrs = ' '.join([
'%s="%s"' % (html_quote(n), html_quote(v))
for n, v in self._html_attrs()])
if attrs:
tag += ' ' + attrs
tag += self._html_extra()
if content is None:
return tag + ' />'
else:
return '%s>%s</%s>' % (tag, content, self.params.get('tag'))
html = property(html__get)
def _html_attrs(self):
return self.attrs.items()
def _html_extra(self):
return ''
def _get_content(self):
"""
Return the content for a tag (for self.html); return None
for an empty tag (like ``<img />``)
"""
raise NotImplementedError
def _add_vars(self, vars):
raise NotImplementedError
def _add_positional(self, args):
raise NotImplementedError
class URL(URLResource):
r"""
>>> u = URL('http://localhost')
>>> u
<URL http://localhost>
>>> u = u['view']
>>> str(u)
'http://localhost/view'
>>> u['//foo'].param(content='view').html
'<a href="http://localhost/view/foo">view</a>'
>>> u.param(confirm='Really?', content='goto').html
'<a href="http://localhost/view" onclick="return confirm(\'Really?\')">goto</a>'
>>> u(title='See "it"', content='goto').html
'<a href="http://localhost/view?title=See+%22it%22">goto</a>'
>>> u('another', var='fuggetaboutit', content='goto').html
'<a href="http://localhost/view/another?var=fuggetaboutit">goto</a>'
>>> u.attr(content='goto').html
Traceback (most recent call last):
....
ValueError: You must give a content param to <URL http://localhost/view attrs(content="goto")> generate anchor tags
>>> str(u['foo=bar%20stuff'])
'http://localhost/view?foo=bar+stuff'
"""
default_params = {'tag': 'a'}
def __str__(self):
return self.href
def _get_content(self):
if not self.params.get('content'):
raise ValueError(
"You must give a content param to %r generate anchor tags"
% self)
return self.params['content']
def _add_vars(self, vars):
url = self
for name in ('confirm', 'content'):
if name in vars:
url = url.param(**{name: vars.pop(name)})
if 'target' in vars:
url = url.attr(target=vars.pop('target'))
return url.var(**vars)
def _add_positional(self, args):
return self.addpath(*args)
def _html_attrs(self):
attrs = self.attrs.items()
attrs.insert(0, ('href', self.href))
if self.params.get('confirm'):
attrs.append(('onclick', 'return confirm(%s)'
% js_repr(self.params['confirm'])))
return attrs
def onclick_goto__get(self):
return 'location.href=%s; return false' % js_repr(self.href)
onclick_goto = property(onclick_goto__get)
def button__get(self):
return self.become(Button)
button = property(button__get)
def js_popup__get(self):
return self.become(JSPopup)
js_popup = property(js_popup__get)
class Image(URLResource):
r"""
>>> i = Image('/images')
>>> i = i / '/foo.png'
>>> i.html
'<img src="/images/foo.png" />'
>>> str(i['alt=foo'])
'<img src="/images/foo.png" alt="foo" />'
>>> i.href
'/images/foo.png'
"""
default_params = {'tag': 'img'}
def __str__(self):
return self.html
def _get_content(self):
return None
def _add_vars(self, vars):
return self.attr(**vars)
def _add_positional(self, args):
return self.addpath(*args)
def _html_attrs(self):
attrs = self.attrs.items()
attrs.insert(0, ('src', self.href))
return attrs
class Button(URLResource):
r"""
>>> u = URL('/')
>>> u = u / 'delete'
>>> b = u.button['confirm=Sure?'](id=5, content='del')
>>> str(b)
'<button onclick="if (confirm(\'Sure?\')) {location.href=\'/delete?id=5\'}; return false">del</button>'
"""
default_params = {'tag': 'button'}
def __str__(self):
return self.html
def _get_content(self):
if self.params.get('content'):
return self.params['content']
if self.attrs.get('value'):
return self.attrs['content']
# @@: Error?
return None
def _add_vars(self, vars):
button = self
if 'confirm' in vars:
button = button.param(confirm=vars.pop('confirm'))
if 'content' in vars:
button = button.param(content=vars.pop('content'))
return button.var(**vars)
def _add_positional(self, args):
return self.addpath(*args)
def _html_attrs(self):
attrs = self.attrs.items()
onclick = 'location.href=%s' % js_repr(self.href)
if self.params.get('confirm'):
onclick = 'if (confirm(%s)) {%s}' % (
js_repr(self.params['confirm']), onclick)
onclick += '; return false'
attrs.insert(0, ('onclick', onclick))
return attrs
class JSPopup(URLResource):
r"""
>>> u = URL('/')
>>> u = u / 'view'
>>> j = u.js_popup(content='view')
>>> j.html
'<a href="/view" onclick="window.open(\'/view\', \'_blank\'); return false" target="_blank">view</a>'
"""
default_params = {'tag': 'a', 'target': '_blank'}
def _add_vars(self, vars):
button = self
for var in ('width', 'height', 'stripped', 'content'):
if var in vars:
button = button.param(**{var: vars.pop(var)})
return button.var(**vars)
def _window_args(self):
p = self.params
features = []
if p.get('stripped'):
p['location'] = p['status'] = p['toolbar'] = '0'
for param in 'channelmode directories fullscreen location menubar resizable scrollbars status titlebar'.split():
if param not in p:
continue
v = p[param]
if v not in ('yes', 'no', '1', '0'):
if v:
v = '1'
else:
v = '0'
features.append('%s=%s' % (param, v))
for param in 'height left top width':
if not p.get(param):
continue
features.append('%s=%s' % (param, p[param]))
args = [self.href, p['target']]
if features:
args.append(','.join(features))
return ', '.join(map(js_repr, args))
def _html_attrs(self):
attrs = self.attrs.items()
onclick = ('window.open(%s); return false'
% self._window_args())
attrs.insert(0, ('target', self.params['target']))
attrs.insert(0, ('onclick', onclick))
attrs.insert(0, ('href', self.href))
return attrs
def _get_content(self):
if not self.params.get('content'):
raise ValueError(
"You must give a content param to %r generate anchor tags"
% self)
return self.params['content']
def _add_positional(self, args):
return self.addpath(*args)
if __name__ == '__main__':
import doctest
doctest.testmod()

View File

@@ -0,0 +1,250 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
"""
Map URL prefixes to WSGI applications. See ``URLMap``
"""
from UserDict import DictMixin
import re
import os
import cgi
from paste import httpexceptions
__all__ = ['URLMap', 'PathProxyURLMap']
def urlmap_factory(loader, global_conf, **local_conf):
if 'not_found_app' in local_conf:
not_found_app = local_conf.pop('not_found_app')
else:
not_found_app = global_conf.get('not_found_app')
if not_found_app:
not_found_app = loader.get_app(not_found_app, global_conf=global_conf)
urlmap = URLMap(not_found_app=not_found_app)
for path, app_name in local_conf.items():
path = parse_path_expression(path)
app = loader.get_app(app_name, global_conf=global_conf)
urlmap[path] = app
return urlmap
def parse_path_expression(path):
"""
Parses a path expression like 'domain foobar.com port 20 /' or
just '/foobar' for a path alone. Returns as an address that
URLMap likes.
"""
parts = path.split()
domain = port = path = None
while parts:
if parts[0] == 'domain':
parts.pop(0)
if not parts:
raise ValueError("'domain' must be followed with a domain name")
if domain:
raise ValueError("'domain' given twice")
domain = parts.pop(0)
elif parts[0] == 'port':
parts.pop(0)
if not parts:
raise ValueError("'port' must be followed with a port number")
if port:
raise ValueError("'port' given twice")
port = parts.pop(0)
else:
if path:
raise ValueError("more than one path given (have %r, got %r)"
% (path, parts[0]))
path = parts.pop(0)
s = ''
if domain:
s = 'http://%s' % domain
if port:
if not domain:
raise ValueError("If you give a port, you must also give a domain")
s += ':' + port
if path:
if s:
s += '/'
s += path
return s
class URLMap(DictMixin):
"""
URLMap instances are dictionary-like object that dispatch to one
of several applications based on the URL.
The dictionary keys are URLs to match (like
``PATH_INFO.startswith(url)``), and the values are applications to
dispatch to. URLs are matched most-specific-first, i.e., longest
URL first. The ``SCRIPT_NAME`` and ``PATH_INFO`` environmental
variables are adjusted to indicate the new context.
URLs can also include domains, like ``http://blah.com/foo``, or as
tuples ``('blah.com', '/foo')``. This will match domain names; without
the ``http://domain`` or with a domain of ``None`` any domain will be
matched (so long as no other explicit domain matches). """
def __init__(self, not_found_app=None):
self.applications = []
if not not_found_app:
not_found_app = self.not_found_app
self.not_found_application = not_found_app
norm_url_re = re.compile('//+')
domain_url_re = re.compile('^(http|https)://')
def not_found_app(self, environ, start_response):
mapper = environ.get('paste.urlmap_object')
if mapper:
matches = [p for p, a in mapper.applications]
extra = 'defined apps: %s' % (
',\n '.join(map(repr, matches)))
else:
extra = ''
extra += '\nSCRIPT_NAME: %r' % environ.get('SCRIPT_NAME')
extra += '\nPATH_INFO: %r' % environ.get('PATH_INFO')
extra += '\nHTTP_HOST: %r' % environ.get('HTTP_HOST')
app = httpexceptions.HTTPNotFound(
environ['PATH_INFO'],
comment=cgi.escape(extra)).wsgi_application
return app(environ, start_response)
def normalize_url(self, url, trim=True):
if isinstance(url, (list, tuple)):
domain = url[0]
url = self.normalize_url(url[1])[1]
return domain, url
assert (not url or url.startswith('/')
or self.domain_url_re.search(url)), (
"URL fragments must start with / or http:// (you gave %r)" % url)
match = self.domain_url_re.search(url)
if match:
url = url[match.end():]
if '/' in url:
domain, url = url.split('/', 1)
url = '/' + url
else:
domain, url = url, ''
else:
domain = None
url = self.norm_url_re.sub('/', url)
if trim:
url = url.rstrip('/')
return domain, url
def sort_apps(self):
"""
Make sure applications are sorted with longest URLs first
"""
def key(app_desc):
(domain, url), app = app_desc
if not domain:
# Make sure empty domains sort last:
return '\xff', -len(url)
else:
return domain, -len(url)
apps = [(key(desc), desc) for desc in self.applications]
apps.sort()
self.applications = [desc for (sortable, desc) in apps]
def __setitem__(self, url, app):
if app is None:
try:
del self[url]
except KeyError:
pass
return
dom_url = self.normalize_url(url)
if dom_url in self:
del self[dom_url]
self.applications.append((dom_url, app))
self.sort_apps()
def __getitem__(self, url):
dom_url = self.normalize_url(url)
for app_url, app in self.applications:
if app_url == dom_url:
return app
raise KeyError(
"No application with the url %r (domain: %r; existing: %s)"
% (url[1], url[0] or '*', self.applications))
def __delitem__(self, url):
url = self.normalize_url(url)
for app_url, app in self.applications:
if app_url == url:
self.applications.remove((app_url, app))
break
else:
raise KeyError(
"No application with the url %r" % (url,))
def keys(self):
return [app_url for app_url, app in self.applications]
def __call__(self, environ, start_response):
host = environ.get('HTTP_HOST', environ.get('SERVER_NAME')).lower()
if ':' in host:
host, port = host.split(':', 1)
else:
if environ['wsgi.url_scheme'] == 'http':
port = '80'
else:
port = '443'
path_info = environ.get('PATH_INFO')
path_info = self.normalize_url(path_info, False)[1]
for (domain, app_url), app in self.applications:
if domain and domain != host and domain != host+':'+port:
continue
if (path_info == app_url
or path_info.startswith(app_url + '/')):
environ['SCRIPT_NAME'] += app_url
environ['PATH_INFO'] = path_info[len(app_url):]
return app(environ, start_response)
environ['paste.urlmap_object'] = self
return self.not_found_application(environ, start_response)
class PathProxyURLMap(object):
"""
This is a wrapper for URLMap that catches any strings that
are passed in as applications; these strings are treated as
filenames (relative to `base_path`) and are passed to the
callable `builder`, which will return an application.
This is intended for cases when configuration files can be
treated as applications.
`base_paste_url` is the URL under which all applications added through
this wrapper must go. Use ``""`` if you want this to not
change incoming URLs.
"""
def __init__(self, map, base_paste_url, base_path, builder):
self.map = map
self.base_paste_url = self.map.normalize_url(base_paste_url)
self.base_path = base_path
self.builder = builder
def __setitem__(self, url, app):
if isinstance(app, (str, unicode)):
app_fn = os.path.join(self.base_path, app)
app = self.builder(app_fn)
url = self.map.normalize_url(url)
# @@: This means http://foo.com/bar will potentially
# match foo.com, but /base_paste_url/bar, which is unintuitive
url = (url[0] or self.base_paste_url[0],
self.base_paste_url[1] + url[1])
self.map[url] = app
def __getattr__(self, attr):
return getattr(self.map, attr)
# This is really the only settable attribute
def not_found_application__get(self):
return self.map.not_found_application
def not_found_application__set(self, value):
self.map.not_found_application = value
not_found_application = property(not_found_application__get,
not_found_application__set)

View File

@@ -0,0 +1,638 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
"""
WSGI applications that parse the URL and dispatch to on-disk resources
"""
import os
import sys
import imp
import mimetypes
try:
import pkg_resources
except ImportError:
pkg_resources = None
from paste import request
from paste import fileapp
from paste.util import import_string
from paste import httpexceptions
from httpheaders import ETAG
from paste.util import converters
class NoDefault(object):
pass
__all__ = ['URLParser', 'StaticURLParser', 'PkgResourcesParser']
class URLParser(object):
"""
WSGI middleware
Application dispatching, based on URL. An instance of `URLParser` is
an application that loads and delegates to other applications. It
looks for files in its directory that match the first part of
PATH_INFO; these may have an extension, but are not required to have
one, in which case the available files are searched to find the
appropriate file. If it is ambiguous, a 404 is returned and an error
logged.
By default there is a constructor for .py files that loads the module,
and looks for an attribute ``application``, which is a ready
application object, or an attribute that matches the module name,
which is a factory for building applications, and is called with no
arguments.
URLParser will also look in __init__.py for special overrides.
These overrides are:
``urlparser_hook(environ)``
This can modify the environment. Its return value is ignored,
and it cannot be used to change the response in any way. You
*can* use this, for example, to manipulate SCRIPT_NAME/PATH_INFO
(try to keep them consistent with the original URL -- but
consuming PATH_INFO and moving that to SCRIPT_NAME is ok).
``urlparser_wrap(environ, start_response, app)``:
After URLParser finds the application, it calls this function
(if present). If this function doesn't call
``app(environ, start_response)`` then the application won't be
called at all! This can be used to allocate resources (with
``try:finally:``) or otherwise filter the output of the
application.
``not_found_hook(environ, start_response)``:
If no file can be found (*in this directory*) to match the
request, then this WSGI application will be called. You can
use this to change the URL and pass the request back to
URLParser again, or on to some other application. This
doesn't catch all ``404 Not Found`` responses, just missing
files.
``application(environ, start_response)``:
This basically overrides URLParser completely, and the given
application is used for all requests. ``urlparser_wrap`` and
``urlparser_hook`` are still called, but the filesystem isn't
searched in any way.
"""
parsers_by_directory = {}
# This is lazily initialized
init_module = NoDefault
global_constructors = {}
def __init__(self, global_conf,
directory, base_python_name,
index_names=NoDefault,
hide_extensions=NoDefault,
ignore_extensions=NoDefault,
constructors=None,
**constructor_conf):
"""
Create a URLParser object that looks at `directory`.
`base_python_name` is the package that this directory
represents, thus any Python modules in this directory will
be given names under this package.
"""
if global_conf:
import warnings
warnings.warn(
'The global_conf argument to URLParser is deprecated; '
'either pass in None or {}, or use make_url_parser',
DeprecationWarning)
else:
global_conf = {}
if os.path.sep != '/':
directory = directory.replace(os.path.sep, '/')
self.directory = directory
self.base_python_name = base_python_name
# This logic here should be deprecated since it is in
# make_url_parser
if index_names is NoDefault:
index_names = global_conf.get(
'index_names', ('index', 'Index', 'main', 'Main'))
self.index_names = converters.aslist(index_names)
if hide_extensions is NoDefault:
hide_extensions = global_conf.get(
'hide_extensions', ('.pyc', '.bak', '.py~', '.pyo'))
self.hide_extensions = converters.aslist(hide_extensions)
if ignore_extensions is NoDefault:
ignore_extensions = global_conf.get(
'ignore_extensions', ())
self.ignore_extensions = converters.aslist(ignore_extensions)
self.constructors = self.global_constructors.copy()
if constructors:
self.constructors.update(constructors)
# @@: Should we also check the global options for constructors?
for name, value in constructor_conf.items():
if not name.startswith('constructor '):
raise ValueError(
"Only extra configuration keys allowed are "
"'constructor .ext = import_expr'; you gave %r "
"(=%r)" % (name, value))
ext = name[len('constructor '):].strip()
if isinstance(value, (str, unicode)):
value = import_string.eval_import(value)
self.constructors[ext] = value
def __call__(self, environ, start_response):
environ['paste.urlparser.base_python_name'] = self.base_python_name
if self.init_module is NoDefault:
self.init_module = self.find_init_module(environ)
path_info = environ.get('PATH_INFO', '')
if not path_info:
return self.add_slash(environ, start_response)
if (self.init_module
and getattr(self.init_module, 'urlparser_hook', None)):
self.init_module.urlparser_hook(environ)
orig_path_info = environ['PATH_INFO']
orig_script_name = environ['SCRIPT_NAME']
application, filename = self.find_application(environ)
if not application:
if (self.init_module
and getattr(self.init_module, 'not_found_hook', None)
and environ.get('paste.urlparser.not_found_parser') is not self):
not_found_hook = self.init_module.not_found_hook
environ['paste.urlparser.not_found_parser'] = self
environ['PATH_INFO'] = orig_path_info
environ['SCRIPT_NAME'] = orig_script_name
return not_found_hook(environ, start_response)
if filename is None:
name, rest_of_path = request.path_info_split(environ['PATH_INFO'])
if not name:
name = 'one of %s' % ', '.join(
self.index_names or
['(no index_names defined)'])
return self.not_found(
environ, start_response,
'Tried to load %s from directory %s'
% (name, self.directory))
else:
environ['wsgi.errors'].write(
'Found resource %s, but could not construct application\n'
% filename)
return self.not_found(
environ, start_response,
'Tried to load %s from directory %s'
% (filename, self.directory))
if (self.init_module
and getattr(self.init_module, 'urlparser_wrap', None)):
return self.init_module.urlparser_wrap(
environ, start_response, application)
else:
return application(environ, start_response)
def find_application(self, environ):
if (self.init_module
and getattr(self.init_module, 'application', None)
and not environ.get('paste.urlparser.init_application') == environ['SCRIPT_NAME']):
environ['paste.urlparser.init_application'] = environ['SCRIPT_NAME']
return self.init_module.application, None
name, rest_of_path = request.path_info_split(environ['PATH_INFO'])
environ['PATH_INFO'] = rest_of_path
if name is not None:
environ['SCRIPT_NAME'] = environ.get('SCRIPT_NAME', '') + '/' + name
if not name:
names = self.index_names
for index_name in names:
filename = self.find_file(environ, index_name)
if filename:
break
else:
# None of the index files found
filename = None
else:
filename = self.find_file(environ, name)
if filename is None:
return None, filename
else:
return self.get_application(environ, filename), filename
def not_found(self, environ, start_response, debug_message=None):
exc = httpexceptions.HTTPNotFound(
'The resource at %s could not be found'
% request.construct_url(environ),
comment=debug_message)
return exc.wsgi_application(environ, start_response)
def add_slash(self, environ, start_response):
"""
This happens when you try to get to a directory
without a trailing /
"""
url = request.construct_url(environ, with_query_string=False)
url += '/'
if environ.get('QUERY_STRING'):
url += '?' + environ['QUERY_STRING']
exc = httpexceptions.HTTPMovedPermanently(
'The resource has moved to %s - you should be redirected '
'automatically.' % url,
headers=[('location', url)])
return exc.wsgi_application(environ, start_response)
def find_file(self, environ, base_filename):
possible = []
"""Cache a few values to reduce function call overhead"""
for filename in os.listdir(self.directory):
base, ext = os.path.splitext(filename)
full_filename = os.path.join(self.directory, filename)
if (ext in self.hide_extensions
or not base):
continue
if filename == base_filename:
possible.append(full_filename)
continue
if ext in self.ignore_extensions:
continue
if base == base_filename:
possible.append(full_filename)
if not possible:
#environ['wsgi.errors'].write(
# 'No file found matching %r in %s\n'
# % (base_filename, self.directory))
return None
if len(possible) > 1:
# If there is an exact match, this isn't 'ambiguous'
# per se; it might mean foo.gif and foo.gif.back for
# instance
if full_filename in possible:
return full_filename
else:
environ['wsgi.errors'].write(
'Ambiguous URL: %s; matches files %s\n'
% (request.construct_url(environ),
', '.join(possible)))
return None
return possible[0]
def get_application(self, environ, filename):
if os.path.isdir(filename):
t = 'dir'
else:
t = os.path.splitext(filename)[1]
constructor = self.constructors.get(t, self.constructors.get('*'))
if constructor is None:
#environ['wsgi.errors'].write(
# 'No constructor found for %s\n' % t)
return constructor
app = constructor(self, environ, filename)
if app is None:
#environ['wsgi.errors'].write(
# 'Constructor %s return None for %s\n' %
# (constructor, filename))
pass
return app
def register_constructor(cls, extension, constructor):
"""
Register a function as a constructor. Registered constructors
apply to all instances of `URLParser`.
The extension should have a leading ``.``, or the special
extensions ``dir`` (for directories) and ``*`` (a catch-all).
`constructor` must be a callable that takes two arguments:
``environ`` and ``filename``, and returns a WSGI application.
"""
d = cls.global_constructors
assert not d.has_key(extension), (
"A constructor already exists for the extension %r (%r) "
"when attemption to register constructor %r"
% (extension, d[extension], constructor))
d[extension] = constructor
register_constructor = classmethod(register_constructor)
def get_parser(self, directory, base_python_name):
"""
Get a parser for the given directory, or create one if
necessary. This way parsers can be cached and reused.
# @@: settings are inherited from the first caller
"""
try:
return self.parsers_by_directory[(directory, base_python_name)]
except KeyError:
parser = self.__class__(
{},
directory, base_python_name,
index_names=self.index_names,
hide_extensions=self.hide_extensions,
ignore_extensions=self.ignore_extensions,
constructors=self.constructors)
self.parsers_by_directory[(directory, base_python_name)] = parser
return parser
def find_init_module(self, environ):
filename = os.path.join(self.directory, '__init__.py')
if not os.path.exists(filename):
return None
return load_module(environ, filename)
def __repr__(self):
return '<%s directory=%r; module=%s at %s>' % (
self.__class__.__name__,
self.directory,
self.base_python_name,
hex(abs(id(self))))
def make_directory(parser, environ, filename):
base_python_name = environ['paste.urlparser.base_python_name']
if base_python_name:
base_python_name += "." + os.path.basename(filename)
else:
base_python_name = os.path.basename(filename)
return parser.get_parser(filename, base_python_name)
URLParser.register_constructor('dir', make_directory)
def make_unknown(parser, environ, filename):
return fileapp.FileApp(filename)
URLParser.register_constructor('*', make_unknown)
def load_module(environ, filename):
base_python_name = environ['paste.urlparser.base_python_name']
module_name = os.path.splitext(os.path.basename(filename))[0]
if base_python_name:
module_name = base_python_name + '.' + module_name
return load_module_from_name(environ, filename, module_name,
environ['wsgi.errors'])
def load_module_from_name(environ, filename, module_name, errors):
if sys.modules.has_key(module_name):
return sys.modules[module_name]
init_filename = os.path.join(os.path.dirname(filename), '__init__.py')
if not os.path.exists(init_filename):
try:
f = open(init_filename, 'w')
except (OSError, IOError), e:
errors.write(
'Cannot write __init__.py file into directory %s (%s)\n'
% (os.path.dirname(filename), e))
return None
f.write('#\n')
f.close()
fp = None
if sys.modules.has_key(module_name):
return sys.modules[module_name]
if '.' in module_name:
parent_name = '.'.join(module_name.split('.')[:-1])
base_name = module_name.split('.')[-1]
parent = load_module_from_name(environ, os.path.dirname(filename),
parent_name, errors)
else:
base_name = module_name
fp = None
try:
fp, pathname, stuff = imp.find_module(
base_name, [os.path.dirname(filename)])
module = imp.load_module(module_name, fp, pathname, stuff)
finally:
if fp is not None:
fp.close()
return module
def make_py(parser, environ, filename):
module = load_module(environ, filename)
if not module:
return None
if hasattr(module, 'application') and module.application:
return getattr(module.application, 'wsgi_application', module.application)
base_name = module.__name__.split('.')[-1]
if hasattr(module, base_name):
obj = getattr(module, base_name)
if hasattr(obj, 'wsgi_application'):
return obj.wsgi_application
else:
# @@: Old behavior; should probably be deprecated eventually:
return getattr(module, base_name)()
environ['wsgi.errors'].write(
"Cound not find application or %s in %s\n"
% (base_name, module))
return None
URLParser.register_constructor('.py', make_py)
class StaticURLParser(object):
"""
Like ``URLParser`` but only serves static files.
``cache_max_age``:
integer specifies Cache-Control max_age in seconds
"""
# @@: Should URLParser subclass from this?
def __init__(self, directory, root_directory=None,
cache_max_age=None):
self.directory = self.normpath(directory)
self.root_directory = self.normpath(root_directory or directory)
self.cache_max_age = cache_max_age
def normpath(path):
return os.path.normcase(os.path.abspath(path))
normpath = staticmethod(normpath)
def __call__(self, environ, start_response):
path_info = environ.get('PATH_INFO', '')
if not path_info:
return self.add_slash(environ, start_response)
if path_info == '/':
# @@: This should obviously be configurable
filename = 'index.html'
else:
filename = request.path_info_pop(environ)
full = self.normpath(os.path.join(self.directory, filename))
if not full.startswith(self.root_directory):
# Out of bounds
return self.not_found(environ, start_response)
if not os.path.exists(full):
return self.not_found(environ, start_response)
if os.path.isdir(full):
# @@: Cache?
return self.__class__(full, root_directory=self.root_directory,
cache_max_age=self.cache_max_age)(environ,
start_response)
if environ.get('PATH_INFO') and environ.get('PATH_INFO') != '/':
return self.error_extra_path(environ, start_response)
if_none_match = environ.get('HTTP_IF_NONE_MATCH')
if if_none_match:
mytime = os.stat(full).st_mtime
if str(mytime) == if_none_match:
headers = []
## FIXME: probably should be
## ETAG.update(headers, '"%s"' % mytime)
ETAG.update(headers, mytime)
start_response('304 Not Modified', headers)
return [''] # empty body
fa = self.make_app(full)
if self.cache_max_age:
fa.cache_control(max_age=self.cache_max_age)
return fa(environ, start_response)
def make_app(self, filename):
return fileapp.FileApp(filename)
def add_slash(self, environ, start_response):
"""
This happens when you try to get to a directory
without a trailing /
"""
url = request.construct_url(environ, with_query_string=False)
url += '/'
if environ.get('QUERY_STRING'):
url += '?' + environ['QUERY_STRING']
exc = httpexceptions.HTTPMovedPermanently(
'The resource has moved to %s - you should be redirected '
'automatically.' % url,
headers=[('location', url)])
return exc.wsgi_application(environ, start_response)
def not_found(self, environ, start_response, debug_message=None):
exc = httpexceptions.HTTPNotFound(
'The resource at %s could not be found'
% request.construct_url(environ),
comment='SCRIPT_NAME=%r; PATH_INFO=%r; looking in %r; debug: %s'
% (environ.get('SCRIPT_NAME'), environ.get('PATH_INFO'),
self.directory, debug_message or '(none)'))
return exc.wsgi_application(environ, start_response)
def error_extra_path(self, environ, start_response):
exc = httpexceptions.HTTPNotFound(
'The trailing path %r is not allowed' % environ['PATH_INFO'])
return exc.wsgi_application(environ, start_response)
def __repr__(self):
return '<%s %r>' % (self.__class__.__name__, self.directory)
def make_static(global_conf, document_root, cache_max_age=None):
"""
Return a WSGI application that serves a directory (configured
with document_root)
cache_max_age - integer specifies CACHE_CONTROL max_age in seconds
"""
if cache_max_age is not None:
cache_max_age = int(cache_max_age)
return StaticURLParser(
document_root, cache_max_age=cache_max_age)
class PkgResourcesParser(StaticURLParser):
def __init__(self, egg_or_spec, resource_name, manager=None, root_resource=None):
if pkg_resources is None:
raise NotImplementedError("This class requires pkg_resources.")
if isinstance(egg_or_spec, (str, unicode)):
self.egg = pkg_resources.get_distribution(egg_or_spec)
else:
self.egg = egg_or_spec
self.resource_name = resource_name
if manager is None:
manager = pkg_resources.ResourceManager()
self.manager = manager
if root_resource is None:
root_resource = resource_name
self.root_resource = os.path.normpath(root_resource)
def __repr__(self):
return '<%s for %s:%r>' % (
self.__class__.__name__,
self.egg.project_name,
self.resource_name)
def __call__(self, environ, start_response):
path_info = environ.get('PATH_INFO', '')
if not path_info:
return self.add_slash(environ, start_response)
if path_info == '/':
# @@: This should obviously be configurable
filename = 'index.html'
else:
filename = request.path_info_pop(environ)
resource = os.path.normcase(os.path.normpath(
self.resource_name + '/' + filename))
if self.root_resource is not None and not resource.startswith(self.root_resource):
# Out of bounds
return self.not_found(environ, start_response)
if not self.egg.has_resource(resource):
return self.not_found(environ, start_response)
if self.egg.resource_isdir(resource):
# @@: Cache?
child_root = self.root_resource is not None and self.root_resource or \
self.resource_name
return self.__class__(self.egg, resource, self.manager,
root_resource=child_root)(environ, start_response)
if environ.get('PATH_INFO') and environ.get('PATH_INFO') != '/':
return self.error_extra_path(environ, start_response)
type, encoding = mimetypes.guess_type(resource)
if not type:
type = 'application/octet-stream'
# @@: I don't know what to do with the encoding.
try:
file = self.egg.get_resource_stream(self.manager, resource)
except (IOError, OSError), e:
exc = httpexceptions.HTTPForbidden(
'You are not permitted to view this file (%s)' % e)
return exc.wsgi_application(environ, start_response)
start_response('200 OK',
[('content-type', type)])
return fileapp._FileIter(file)
def not_found(self, environ, start_response, debug_message=None):
exc = httpexceptions.HTTPNotFound(
'The resource at %s could not be found'
% request.construct_url(environ),
comment='SCRIPT_NAME=%r; PATH_INFO=%r; looking in egg:%s#%r; debug: %s'
% (environ.get('SCRIPT_NAME'), environ.get('PATH_INFO'),
self.egg, self.resource_name, debug_message or '(none)'))
return exc.wsgi_application(environ, start_response)
def make_pkg_resources(global_conf, egg, resource_name=''):
"""
A static file parser that loads data from an egg using
``pkg_resources``. Takes a configuration value ``egg``, which is
an egg spec, and a base ``resource_name`` (default empty string)
which is the path in the egg that this starts at.
"""
if pkg_resources is None:
raise NotImplementedError("This function requires pkg_resources.")
return PkgResourcesParser(egg, resource_name)
def make_url_parser(global_conf, directory, base_python_name,
index_names=None, hide_extensions=None,
ignore_extensions=None,
**constructor_conf):
"""
Create a URLParser application that looks in ``directory``, which
should be the directory for the Python package named in
``base_python_name``. ``index_names`` are used when viewing the
directory (like ``'index'`` for ``'index.html'``).
``hide_extensions`` are extensions that are not viewable (like
``'.pyc'``) and ``ignore_extensions`` are viewable but only if an
explicit extension is given.
"""
if index_names is None:
index_names = global_conf.get(
'index_names', ('index', 'Index', 'main', 'Main'))
index_names = converters.aslist(index_names)
if hide_extensions is None:
hide_extensions = global_conf.get(
'hide_extensions', ('.pyc', 'bak', 'py~'))
hide_extensions = converters.aslist(hide_extensions)
if ignore_extensions is None:
ignore_extensions = global_conf.get(
'ignore_extensions', ())
ignore_extensions = converters.aslist(ignore_extensions)
# There's no real way to set constructors currently...
return URLParser({}, directory, base_python_name,
index_names=index_names,
hide_extensions=hide_extensions,
ignore_extensions=ignore_extensions,
**constructor_conf)

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,167 @@
"""A more or less complete user-defined wrapper around dictionary objects."""
class UserDict:
def __init__(self, dict=None, **kwargs):
self.data = {}
if dict is not None:
if not hasattr(dict,'keys'):
dict = type({})(dict) # make mapping from a sequence
self.update(dict)
if len(kwargs):
self.update(kwargs)
def __repr__(self): return repr(self.data)
def __cmp__(self, dict):
if isinstance(dict, UserDict):
return cmp(self.data, dict.data)
else:
return cmp(self.data, dict)
def __len__(self): return len(self.data)
def __getitem__(self, key): return self.data[key]
def __setitem__(self, key, item): self.data[key] = item
def __delitem__(self, key): del self.data[key]
def clear(self): self.data.clear()
def copy(self):
if self.__class__ is UserDict:
return UserDict(self.data)
import copy
data = self.data
try:
self.data = {}
c = copy.copy(self)
finally:
self.data = data
c.update(self)
return c
def keys(self): return self.data.keys()
def items(self): return self.data.items()
def iteritems(self): return self.data.iteritems()
def iterkeys(self): return self.data.iterkeys()
def itervalues(self): return self.data.itervalues()
def values(self): return self.data.values()
def has_key(self, key): return self.data.has_key(key)
def update(self, dict):
if isinstance(dict, UserDict):
self.data.update(dict.data)
elif isinstance(dict, type(self.data)):
self.data.update(dict)
else:
for k, v in dict.items():
self[k] = v
def get(self, key, failobj=None):
if not self.has_key(key):
return failobj
return self[key]
def setdefault(self, key, failobj=None):
if not self.has_key(key):
self[key] = failobj
return self[key]
def pop(self, key, *args):
return self.data.pop(key, *args)
def popitem(self):
return self.data.popitem()
def __contains__(self, key):
return key in self.data
def fromkeys(cls, iterable, value=None):
d = cls()
for key in iterable:
d[key] = value
return d
fromkeys = classmethod(fromkeys)
class IterableUserDict(UserDict):
def __iter__(self):
return iter(self.data)
class DictMixin:
# Mixin defining all dictionary methods for classes that already have
# a minimum dictionary interface including getitem, setitem, delitem,
# and keys. Without knowledge of the subclass constructor, the mixin
# does not define __init__() or copy(). In addition to the four base
# methods, progressively more efficiency comes with defining
# __contains__(), __iter__(), and iteritems().
# second level definitions support higher levels
def __iter__(self):
for k in self.keys():
yield k
def has_key(self, key):
try:
value = self[key]
except KeyError:
return False
return True
def __contains__(self, key):
return self.has_key(key)
# third level takes advantage of second level definitions
def iteritems(self):
for k in self:
yield (k, self[k])
def iterkeys(self):
return self.__iter__()
# fourth level uses definitions from lower levels
def itervalues(self):
for _, v in self.iteritems():
yield v
def values(self):
return [v for _, v in self.iteritems()]
def items(self):
return list(self.iteritems())
def clear(self):
for key in self.keys():
del self[key]
def setdefault(self, key, default):
try:
return self[key]
except KeyError:
self[key] = default
return default
def pop(self, key, *args):
if len(args) > 1:
raise TypeError, "pop expected at most 2 arguments, got "\
+ repr(1 + len(args))
try:
value = self[key]
except KeyError:
if args:
return args[0]
raise
del self[key]
return value
def popitem(self):
try:
k, v = self.iteritems().next()
except StopIteration:
raise KeyError, 'container is empty'
del self[k]
return (k, v)
def update(self, other):
# Make progressively weaker assumptions about "other"
if hasattr(other, 'iteritems'): # iteritems saves memory and lookups
for k, v in other.iteritems():
self[k] = v
elif hasattr(other, '__iter__'): # iter saves memory
for k in other:
self[k] = other[k]
else:
for k in other.keys():
self[k] = other[k]
def get(self, key, default=None):
try:
return self[key]
except KeyError:
return default
def __repr__(self):
return repr(dict(self.iteritems()))
def __cmp__(self, other):
if other is None:
return 1
if isinstance(other, DictMixin):
other = dict(other.iteritems())
return cmp(dict(self.iteritems()), other)
def __len__(self):
return len(self.keys())
def __nonzero__(self):
return bool(self.iteritems())

View File

@@ -0,0 +1,4 @@
"""
Package for miscellaneous routines that do not depend on other parts
of Paste
"""

View File

@@ -0,0 +1,42 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
class ClassInitMeta(type):
def __new__(meta, class_name, bases, new_attrs):
cls = type.__new__(meta, class_name, bases, new_attrs)
if (new_attrs.has_key('__classinit__')
and not isinstance(cls.__classinit__, staticmethod)):
setattr(cls, '__classinit__',
staticmethod(cls.__classinit__.im_func))
if hasattr(cls, '__classinit__'):
cls.__classinit__(cls, new_attrs)
return cls
def build_properties(cls, new_attrs):
"""
Given a class and a new set of attributes (as passed in by
__classinit__), create or modify properties based on functions
with special names ending in __get, __set, and __del.
"""
for name, value in new_attrs.items():
if (name.endswith('__get') or name.endswith('__set')
or name.endswith('__del')):
base = name[:-5]
if hasattr(cls, base):
old_prop = getattr(cls, base)
if not isinstance(old_prop, property):
raise ValueError(
"Attribute %s is a %s, not a property; function %s is named like a property"
% (base, type(old_prop), name))
attrs = {'fget': old_prop.fget,
'fset': old_prop.fset,
'fdel': old_prop.fdel,
'doc': old_prop.__doc__}
else:
attrs = {}
attrs['f' + name[-3:]] = value
if name.endswith('__get') and value.__doc__:
attrs['doc'] = value.__doc__
new_prop = property(**attrs)
setattr(cls, base, new_prop)

View File

@@ -0,0 +1,38 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
class classinstancemethod(object):
"""
Acts like a class method when called from a class, like an
instance method when called by an instance. The method should
take two arguments, 'self' and 'cls'; one of these will be None
depending on how the method was called.
"""
def __init__(self, func):
self.func = func
self.__doc__ = func.__doc__
def __get__(self, obj, type=None):
return _methodwrapper(self.func, obj=obj, type=type)
class _methodwrapper(object):
def __init__(self, func, obj, type):
self.func = func
self.obj = obj
self.type = type
def __call__(self, *args, **kw):
assert not kw.has_key('self') and not kw.has_key('cls'), (
"You cannot use 'self' or 'cls' arguments to a "
"classinstancemethod")
return self.func(*((self.obj, self.type) + args), **kw)
def __repr__(self):
if self.obj is None:
return ('<bound class method %s.%s>'
% (self.type.__name__, self.func.func_name))
else:
return ('<bound method %s.%s of %r>'
% (self.type.__name__, self.func.func_name, self.obj))

View File

@@ -0,0 +1,26 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
def asbool(obj):
if isinstance(obj, (str, unicode)):
obj = obj.strip().lower()
if obj in ['true', 'yes', 'on', 'y', 't', '1']:
return True
elif obj in ['false', 'no', 'off', 'n', 'f', '0']:
return False
else:
raise ValueError(
"String is not true/false: %r" % obj)
return bool(obj)
def aslist(obj, sep=None, strip=True):
if isinstance(obj, (str, unicode)):
lst = obj.split(sep)
if strip:
lst = [v.strip() for v in lst]
return lst
elif isinstance(obj, (list, tuple)):
return obj
elif obj is None:
return []
else:
return [obj]

View File

@@ -0,0 +1,103 @@
"""
DateInterval.py
Convert interval strings (in the form of 1w2d, etc) to
seconds, and back again. Is not exactly about months or
years (leap years in particular).
Accepts (y)ear, (b)month, (w)eek, (d)ay, (h)our, (m)inute, (s)econd.
Exports only timeEncode and timeDecode functions.
"""
import re
__all__ = ['interval_decode', 'interval_encode']
second = 1
minute = second*60
hour = minute*60
day = hour*24
week = day*7
month = day*30
year = day*365
timeValues = {
'y': year,
'b': month,
'w': week,
'd': day,
'h': hour,
'm': minute,
's': second,
}
timeOrdered = timeValues.items()
timeOrdered.sort(lambda a, b: -cmp(a[1], b[1]))
def interval_encode(seconds, include_sign=False):
"""Encodes a number of seconds (representing a time interval)
into a form like 1h2d3s.
>>> interval_encode(10)
'10s'
>>> interval_encode(493939)
'5d17h12m19s'
"""
s = ''
orig = seconds
seconds = abs(seconds)
for char, amount in timeOrdered:
if seconds >= amount:
i, seconds = divmod(seconds, amount)
s += '%i%s' % (i, char)
if orig < 0:
s = '-' + s
elif not orig:
return '0'
elif include_sign:
s = '+' + s
return s
_timeRE = re.compile(r'[0-9]+[a-zA-Z]')
def interval_decode(s):
"""Decodes a number in the format 1h4d3m (1 hour, 3 days, 3 minutes)
into a number of seconds
>>> interval_decode('40s')
40
>>> interval_decode('10000s')
10000
>>> interval_decode('3d1w45s')
864045
"""
time = 0
sign = 1
s = s.strip()
if s.startswith('-'):
s = s[1:]
sign = -1
elif s.startswith('+'):
s = s[1:]
for match in allMatches(s, _timeRE):
char = match.group(0)[-1].lower()
if not timeValues.has_key(char):
# @@: should signal error
continue
time += int(match.group(0)[:-1]) * timeValues[char]
return time
# @@-sgd 2002-12-23 - this function does not belong in this module, find a better place.
def allMatches(source, regex):
"""Return a list of matches for regex in source
"""
pos = 0
end = len(source)
rv = []
match = regex.search(source, pos)
while match:
rv.append(match)
match = regex.search(source, match.end() )
return rv
if __name__ == '__main__':
import doctest
doctest.testmod()

View File

@@ -0,0 +1,361 @@
# (c) 2005 Clark C. Evans and contributors
# This module is part of the Python Paste Project and is released under
# the MIT License: http://www.opensource.org/licenses/mit-license.php
# Some of this code was funded by: http://prometheusresearch.com
"""
Date, Time, and Timespan Parsing Utilities
This module contains parsing support to create "human friendly"
``datetime`` object parsing. The explicit goal of these routines is
to provide a multi-format date/time support not unlike that found in
Microsoft Excel. In most approaches, the input is very "strict" to
prevent errors -- however, this approach is much more liberal since we
are assuming the user-interface is parroting back the normalized value
and thus the user has immediate feedback if the data is not typed in
correctly.
``parse_date`` and ``normalize_date``
These functions take a value like '9 jan 2007' and returns either an
``date`` object, or an ISO 8601 formatted date value such
as '2007-01-09'. There is an option to provide an Oracle database
style output as well, ``09 JAN 2007``, but this is not the default.
This module always treats '/' delimiters as using US date order
(since the author's clients are US based), hence '1/9/2007' is
January 9th. Since this module treats the '-' as following
European order this supports both modes of data-entry; together
with immediate parroting back the result to the screen, the author
has found this approach to work well in pratice.
``parse_time`` and ``normalize_time``
These functions take a value like '1 pm' and returns either an
``time`` object, or an ISO 8601 formatted 24h clock time
such as '13:00'. There is an option to provide for US style time
values, '1:00 PM', however this is not the default.
``parse_datetime`` and ``normalize_datetime``
These functions take a value like '9 jan 2007 at 1 pm' and returns
either an ``datetime`` object, or an ISO 8601 formatted
return (without the T) such as '2007-01-09 13:00'. There is an
option to provide for Oracle / US style, '09 JAN 2007 @ 1:00 PM',
however this is not the default.
``parse_delta`` and ``normalize_delta``
These functions take a value like '1h 15m' and returns either an
``timedelta`` object, or an 2-decimal fixed-point
numerical value in hours, such as '1.25'. The rationale is to
support meeting or time-billing lengths, not to be an accurate
representation in mili-seconds. As such not all valid
``timedelta`` values will have a normalized representation.
"""
from datetime import timedelta, time, date
from time import localtime
import string
__all__ = ['parse_timedelta', 'normalize_timedelta',
'parse_time', 'normalize_time',
'parse_date', 'normalize_date']
def _number(val):
try:
return string.atoi(val)
except:
return None
#
# timedelta
#
def parse_timedelta(val):
"""
returns a ``timedelta`` object, or None
"""
if not val:
return None
val = string.lower(val)
if "." in val:
val = float(val)
return timedelta(hours=int(val), minutes=60*(val % 1.0))
fHour = ("h" in val or ":" in val)
fMin = ("m" in val or ":" in val)
fFraction = "." in val
for noise in "minu:teshour()":
val = string.replace(val, noise, ' ')
val = string.strip(val)
val = string.split(val)
hr = 0.0
mi = 0
val.reverse()
if fHour:
hr = int(val.pop())
if fMin:
mi = int(val.pop())
if len(val) > 0 and not hr:
hr = int(val.pop())
return timedelta(hours=hr, minutes=mi)
def normalize_timedelta(val):
"""
produces a normalized string value of the timedelta
This module returns a normalized time span value consisting of the
number of hours in fractional form. For example '1h 15min' is
formatted as 01.25.
"""
if type(val) == str:
val = parse_timedelta(val)
if not val:
return ''
hr = val.seconds/3600
mn = (val.seconds % 3600)/60
return "%d.%02d" % (hr, mn * 100/60)
#
# time
#
def parse_time(val):
if not val:
return None
hr = mi = 0
val = string.lower(val)
amflag = (-1 != string.find(val, 'a')) # set if AM is found
pmflag = (-1 != string.find(val, 'p')) # set if PM is found
for noise in ":amp.":
val = string.replace(val, noise, ' ')
val = string.split(val)
if len(val) > 1:
hr = int(val[0])
mi = int(val[1])
else:
val = val[0]
if len(val) < 1:
pass
elif 'now' == val:
tm = localtime()
hr = tm[3]
mi = tm[4]
elif 'noon' == val:
hr = 12
elif len(val) < 3:
hr = int(val)
if not amflag and not pmflag and hr < 7:
hr += 12
elif len(val) < 5:
hr = int(val[:-2])
mi = int(val[-2:])
else:
hr = int(val[:1])
if amflag and hr >= 12:
hr = hr - 12
if pmflag and hr < 12:
hr = hr + 12
return time(hr, mi)
def normalize_time(value, ampm):
if not value:
return ''
if type(value) == str:
value = parse_time(value)
if not ampm:
return "%02d:%02d" % (value.hour, value.minute)
hr = value.hour
am = "AM"
if hr < 1 or hr > 23:
hr = 12
elif hr >= 12:
am = "PM"
if hr > 12:
hr = hr - 12
return "%02d:%02d %s" % (hr, value.minute, am)
#
# Date Processing
#
_one_day = timedelta(days=1)
_str2num = {'jan':1, 'feb':2, 'mar':3, 'apr':4, 'may':5, 'jun':6,
'jul':7, 'aug':8, 'sep':9, 'oct':10, 'nov':11, 'dec':12 }
def _month(val):
for (key, mon) in _str2num.items():
if key in val:
return mon
raise TypeError("unknown month '%s'" % val)
_days_in_month = {1: 31, 2: 28, 3: 31, 4: 30, 5: 31, 6: 30,
7: 31, 8: 31, 9: 30, 10: 31, 11: 30, 12: 31,
}
_num2str = {1: 'Jan', 2: 'Feb', 3: 'Mar', 4: 'Apr', 5: 'May', 6: 'Jun',
7: 'Jul', 8: 'Aug', 9: 'Sep', 10: 'Oct', 11: 'Nov', 12: 'Dec',
}
_wkdy = ("mon", "tue", "wed", "thu", "fri", "sat", "sun")
def parse_date(val):
if not(val):
return None
val = string.lower(val)
now = None
# optimized check for YYYY-MM-DD
strict = val.split("-")
if len(strict) == 3:
(y, m, d) = strict
if "+" in d:
d = d.split("+")[0]
if " " in d:
d = d.split(" ")[0]
try:
now = date(int(y), int(m), int(d))
val = "xxx" + val[10:]
except ValueError:
pass
# allow for 'now', 'mon', 'tue', etc.
if not now:
chk = val[:3]
if chk in ('now','tod'):
now = date.today()
elif chk in _wkdy:
now = date.today()
idx = list(_wkdy).index(chk) + 1
while now.isoweekday() != idx:
now += _one_day
# allow dates to be modified via + or - /w number of days, so
# that now+3 is three days from now
if now:
tail = val[3:].strip()
tail = tail.replace("+"," +").replace("-"," -")
for item in tail.split():
try:
days = int(item)
except ValueError:
pass
else:
now += timedelta(days=days)
return now
# ok, standard parsing
yr = mo = dy = None
for noise in ('/', '-', ',', '*'):
val = string.replace(val, noise, ' ')
for noise in _wkdy:
val = string.replace(val, noise, ' ')
out = []
last = False
ldig = False
for ch in val:
if ch.isdigit():
if last and not ldig:
out.append(' ')
last = ldig = True
else:
if ldig:
out.append(' ')
ldig = False
last = True
out.append(ch)
val = string.split("".join(out))
if 3 == len(val):
a = _number(val[0])
b = _number(val[1])
c = _number(val[2])
if len(val[0]) == 4:
yr = a
if b: # 1999 6 23
mo = b
dy = c
else: # 1999 Jun 23
mo = _month(val[1])
dy = c
elif a > 0:
yr = c
if len(val[2]) < 4:
raise TypeError("four digit year required")
if b: # 6 23 1999
dy = b
mo = a
else: # 23 Jun 1999
dy = a
mo = _month(val[1])
else: # Jun 23, 2000
dy = b
yr = c
if len(val[2]) < 4:
raise TypeError("four digit year required")
mo = _month(val[0])
elif 2 == len(val):
a = _number(val[0])
b = _number(val[1])
if a > 999:
yr = a
dy = 1
if b > 0: # 1999 6
mo = b
else: # 1999 Jun
mo = _month(val[1])
elif a > 0:
if b > 999: # 6 1999
mo = a
yr = b
dy = 1
elif b > 0: # 6 23
mo = a
dy = b
else: # 23 Jun
dy = a
mo = _month(val[1])
else:
if b > 999: # Jun 2001
yr = b
dy = 1
else: # Jun 23
dy = b
mo = _month(val[0])
elif 1 == len(val):
val = val[0]
if not val.isdigit():
mo = _month(val)
if mo is not None:
dy = 1
else:
v = _number(val)
val = str(v)
if 8 == len(val): # 20010623
yr = _number(val[:4])
mo = _number(val[4:6])
dy = _number(val[6:])
elif len(val) in (3,4):
if v > 1300: # 2004
yr = v
mo = 1
dy = 1
else: # 1202
mo = _number(val[:-2])
dy = _number(val[-2:])
elif v < 32:
dy = v
else:
raise TypeError("four digit year required")
tm = localtime()
if mo is None:
mo = tm[1]
if dy is None:
dy = tm[2]
if yr is None:
yr = tm[0]
return date(yr, mo, dy)
def normalize_date(val, iso8601=True):
if not val:
return ''
if type(val) == str:
val = parse_date(val)
if iso8601:
return "%4d-%02d-%02d" % (val.year, val.month, val.day)
return "%02d %s %4d" % (val.day, _num2str[val.month], val.year)

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,53 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
class FileMixin(object):
"""
Used to provide auxiliary methods to objects simulating files.
Objects must implement write, and read if they are input files.
Also they should implement close.
Other methods you may wish to override:
* flush()
* seek(offset[, whence])
* tell()
* truncate([size])
Attributes you may wish to provide:
* closed
* encoding (you should also respect that in write())
* mode
* newlines (hard to support)
* softspace
"""
def flush(self):
pass
def next(self):
return self.readline()
def readline(self, size=None):
# @@: This is a lame implementation; but a buffer would probably
# be necessary for a better implementation
output = []
while 1:
next = self.read(1)
if not next:
return ''.join(output)
output.append(next)
if size and size > 0 and len(output) >= size:
return ''.join(output)
if next == '\n':
# @@: also \r?
return ''.join(output)
def xreadlines(self):
return self
def writelines(self, lines):
for line in lines:
self.write(line)

View File

@@ -0,0 +1,99 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
# Note: you may want to copy this into your setup.py file verbatim, as
# you can't import this from another package, when you don't know if
# that package is installed yet.
import os
import sys
from fnmatch import fnmatchcase
from distutils.util import convert_path
# Provided as an attribute, so you can append to these instead
# of replicating them:
standard_exclude = ('*.py', '*.pyc', '*$py.class', '*~', '.*', '*.bak')
standard_exclude_directories = ('.*', 'CVS', '_darcs', './build',
'./dist', 'EGG-INFO', '*.egg-info')
def find_package_data(
where='.', package='',
exclude=standard_exclude,
exclude_directories=standard_exclude_directories,
only_in_packages=True,
show_ignored=False):
"""
Return a dictionary suitable for use in ``package_data``
in a distutils ``setup.py`` file.
The dictionary looks like::
{'package': [files]}
Where ``files`` is a list of all the files in that package that
don't match anything in ``exclude``.
If ``only_in_packages`` is true, then top-level directories that
are not packages won't be included (but directories under packages
will).
Directories matching any pattern in ``exclude_directories`` will
be ignored; by default directories with leading ``.``, ``CVS``,
and ``_darcs`` will be ignored.
If ``show_ignored`` is true, then all the files that aren't
included in package data are shown on stderr (for debugging
purposes).
Note patterns use wildcards, or can be exact paths (including
leading ``./``), and all searching is case-insensitive.
"""
out = {}
stack = [(convert_path(where), '', package, only_in_packages)]
while stack:
where, prefix, package, only_in_packages = stack.pop(0)
for name in os.listdir(where):
fn = os.path.join(where, name)
if os.path.isdir(fn):
bad_name = False
for pattern in exclude_directories:
if (fnmatchcase(name, pattern)
or fn.lower() == pattern.lower()):
bad_name = True
if show_ignored:
print >> sys.stderr, (
"Directory %s ignored by pattern %s"
% (fn, pattern))
break
if bad_name:
continue
if (os.path.isfile(os.path.join(fn, '__init__.py'))
and not prefix):
if not package:
new_package = name
else:
new_package = package + '.' + name
stack.append((fn, '', new_package, False))
else:
stack.append((fn, prefix + name + '/', package, only_in_packages))
elif package or not only_in_packages:
# is a file
bad_name = False
for pattern in exclude:
if (fnmatchcase(name, pattern)
or fn.lower() == pattern.lower()):
bad_name = True
if show_ignored:
print >> sys.stderr, (
"File %s ignored by pattern %s"
% (fn, pattern))
break
if bad_name:
continue
out.setdefault(package, []).append(prefix+name)
return out
if __name__ == '__main__':
import pprint
pprint.pprint(
find_package_data(show_ignored=True))

View File

@@ -0,0 +1,26 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
import sys
import os
def find_package(dir):
"""
Given a directory, finds the equivalent package name. If it
is directly in sys.path, returns ''.
"""
dir = os.path.abspath(dir)
orig_dir = dir
path = map(os.path.abspath, sys.path)
packages = []
last_dir = None
while 1:
if dir in path:
return '.'.join(packages)
packages.insert(0, os.path.basename(dir))
dir = os.path.dirname(dir)
if last_dir == dir:
raise ValueError(
"%s is not under any path found in sys.path" % orig_dir)
last_dir = dir

View File

@@ -0,0 +1,95 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
"""
'imports' a string -- converts a string to a Python object, importing
any necessary modules and evaluating the expression. Everything
before the : in an import expression is the module path; everything
after is an expression to be evaluated in the namespace of that
module.
Alternately, if no : is present, then import the modules and get the
attributes as necessary. Arbitrary expressions are not allowed in
that case.
"""
def eval_import(s):
"""
Import a module, or import an object from a module.
A module name like ``foo.bar:baz()`` can be used, where
``foo.bar`` is the module, and ``baz()`` is an expression
evaluated in the context of that module. Note this is not safe on
arbitrary strings because of the eval.
"""
if ':' not in s:
return simple_import(s)
module_name, expr = s.split(':', 1)
module = import_module(module_name)
obj = eval(expr, module.__dict__)
return obj
def simple_import(s):
"""
Import a module, or import an object from a module.
A name like ``foo.bar.baz`` can be a module ``foo.bar.baz`` or a
module ``foo.bar`` with an object ``baz`` in it, or a module
``foo`` with an object ``bar`` with an attribute ``baz``.
"""
parts = s.split('.')
module = import_module(parts[0])
name = parts[0]
parts = parts[1:]
last_import_error = None
while parts:
name += '.' + parts[0]
try:
module = import_module(name)
parts = parts[1:]
except ImportError, e:
last_import_error = e
break
obj = module
while parts:
try:
obj = getattr(module, parts[0])
except AttributeError:
raise ImportError(
"Cannot find %s in module %r (stopped importing modules with error %s)" % (parts[0], module, last_import_error))
parts = parts[1:]
return obj
def import_module(s):
"""
Import a module.
"""
mod = __import__(s)
parts = s.split('.')
for part in parts[1:]:
mod = getattr(mod, part)
return mod
def try_import_module(module_name):
"""
Imports a module, but catches import errors. Only catches errors
when that module doesn't exist; if that module itself has an
import error it will still get raised. Returns None if the module
doesn't exist.
"""
try:
return import_module(module_name)
except ImportError, e:
if not getattr(e, 'args', None):
raise
desc = e.args[0]
if not desc.startswith('No module named '):
raise
desc = desc[len('No module named '):]
# If you import foo.bar.baz, the bad import could be any
# of foo.bar.baz, bar.baz, or baz; we'll test them all:
parts = module_name.split('.')
for i in range(len(parts)):
if desc == '.'.join(parts[i:]):
return None
raise

View File

@@ -0,0 +1,511 @@
# -*- coding: iso-8859-15 -*-
"""Immutable integer set type.
Integer set class.
Copyright (C) 2006, Heiko Wundram.
Released under the MIT license.
"""
# Version information
# -------------------
__author__ = "Heiko Wundram <me@modelnine.org>"
__version__ = "0.2"
__revision__ = "6"
__date__ = "2006-01-20"
# Utility classes
# ---------------
class _Infinity(object):
"""Internal type used to represent infinity values."""
__slots__ = ["_neg"]
def __init__(self,neg):
self._neg = neg
def __lt__(self,value):
if not isinstance(value,(int,long,_Infinity)):
return NotImplemented
return ( self._neg and
not ( isinstance(value,_Infinity) and value._neg ) )
def __le__(self,value):
if not isinstance(value,(int,long,_Infinity)):
return NotImplemented
return self._neg
def __gt__(self,value):
if not isinstance(value,(int,long,_Infinity)):
return NotImplemented
return not ( self._neg or
( isinstance(value,_Infinity) and not value._neg ) )
def __ge__(self,value):
if not isinstance(value,(int,long,_Infinity)):
return NotImplemented
return not self._neg
def __eq__(self,value):
if not isinstance(value,(int,long,_Infinity)):
return NotImplemented
return isinstance(value,_Infinity) and self._neg == value._neg
def __ne__(self,value):
if not isinstance(value,(int,long,_Infinity)):
return NotImplemented
return not isinstance(value,_Infinity) or self._neg <> value._neg
def __repr__(self):
return "None"
# Constants
# ---------
_MININF = _Infinity(True)
_MAXINF = _Infinity(False)
# Integer set class
# -----------------
class IntSet(object):
"""Integer set class with efficient storage in a RLE format of ranges.
Supports minus and plus infinity in the range."""
__slots__ = ["_ranges","_min","_max","_hash"]
def __init__(self,*args,**kwargs):
"""Initialize an integer set. The constructor accepts an unlimited
number of arguments that may either be tuples in the form of
(start,stop) where either start or stop may be a number or None to
represent maximum/minimum in that direction. The range specified by
(start,stop) is always inclusive (differing from the builtin range
operator).
Keyword arguments that can be passed to an integer set are min and
max, which specify the minimum and maximum number in the set,
respectively. You can also pass None here to represent minus or plus
infinity, which is also the default.
"""
# Special case copy constructor.
if len(args) == 1 and isinstance(args[0],IntSet):
if kwargs:
raise ValueError("No keyword arguments for copy constructor.")
self._min = args[0]._min
self._max = args[0]._max
self._ranges = args[0]._ranges
self._hash = args[0]._hash
return
# Initialize set.
self._ranges = []
# Process keyword arguments.
self._min = kwargs.pop("min",_MININF)
self._max = kwargs.pop("max",_MAXINF)
if self._min is None:
self._min = _MININF
if self._max is None:
self._max = _MAXINF
# Check keyword arguments.
if kwargs:
raise ValueError("Invalid keyword argument.")
if not ( isinstance(self._min,(int,long)) or self._min is _MININF ):
raise TypeError("Invalid type of min argument.")
if not ( isinstance(self._max,(int,long)) or self._max is _MAXINF ):
raise TypeError("Invalid type of max argument.")
if ( self._min is not _MININF and self._max is not _MAXINF and
self._min > self._max ):
raise ValueError("Minimum is not smaller than maximum.")
if isinstance(self._max,(int,long)):
self._max += 1
# Process arguments.
for arg in args:
if isinstance(arg,(int,long)):
start, stop = arg, arg+1
elif isinstance(arg,tuple):
if len(arg) <> 2:
raise ValueError("Invalid tuple, must be (start,stop).")
# Process argument.
start, stop = arg
if start is None:
start = self._min
if stop is None:
stop = self._max
# Check arguments.
if not ( isinstance(start,(int,long)) or start is _MININF ):
raise TypeError("Invalid type of tuple start.")
if not ( isinstance(stop,(int,long)) or stop is _MAXINF ):
raise TypeError("Invalid type of tuple stop.")
if ( start is not _MININF and stop is not _MAXINF and
start > stop ):
continue
if isinstance(stop,(int,long)):
stop += 1
else:
raise TypeError("Invalid argument.")
if start > self._max:
continue
elif start < self._min:
start = self._min
if stop < self._min:
continue
elif stop > self._max:
stop = self._max
self._ranges.append((start,stop))
# Normalize set.
self._normalize()
# Utility functions for set operations
# ------------------------------------
def _iterranges(self,r1,r2,minval=_MININF,maxval=_MAXINF):
curval = minval
curstates = {"r1":False,"r2":False}
imax, jmax = 2*len(r1), 2*len(r2)
i, j = 0, 0
while i < imax or j < jmax:
if i < imax and ( ( j < jmax and
r1[i>>1][i&1] < r2[j>>1][j&1] ) or
j == jmax ):
cur_r, newname, newstate = r1[i>>1][i&1], "r1", not (i&1)
i += 1
else:
cur_r, newname, newstate = r2[j>>1][j&1], "r2", not (j&1)
j += 1
if curval < cur_r:
if cur_r > maxval:
break
yield curstates, (curval,cur_r)
curval = cur_r
curstates[newname] = newstate
if curval < maxval:
yield curstates, (curval,maxval)
def _normalize(self):
self._ranges.sort()
i = 1
while i < len(self._ranges):
if self._ranges[i][0] < self._ranges[i-1][1]:
self._ranges[i-1] = (self._ranges[i-1][0],
max(self._ranges[i-1][1],
self._ranges[i][1]))
del self._ranges[i]
else:
i += 1
self._ranges = tuple(self._ranges)
self._hash = hash(self._ranges)
def __coerce__(self,other):
if isinstance(other,IntSet):
return self, other
elif isinstance(other,(int,long,tuple)):
try:
return self, self.__class__(other)
except TypeError:
# Catch a type error, in that case the structure specified by
# other is something we can't coerce, return NotImplemented.
# ValueErrors are not caught, they signal that the data was
# invalid for the constructor. This is appropriate to signal
# as a ValueError to the caller.
return NotImplemented
elif isinstance(other,list):
try:
return self, self.__class__(*other)
except TypeError:
# See above.
return NotImplemented
return NotImplemented
# Set function definitions
# ------------------------
def _make_function(name,type,doc,pall,pany=None):
"""Makes a function to match two ranges. Accepts two types: either
'set', which defines a function which returns a set with all ranges
matching pall (pany is ignored), or 'bool', which returns True if pall
matches for all ranges and pany matches for any one range. doc is the
dostring to give this function. pany may be none to ignore the any
match.
The predicates get a dict with two keys, 'r1', 'r2', which denote
whether the current range is present in range1 (self) and/or range2
(other) or none of the two, respectively."""
if type == "set":
def f(self,other):
coerced = self.__coerce__(other)
if coerced is NotImplemented:
return NotImplemented
other = coerced[1]
newset = self.__class__.__new__(self.__class__)
newset._min = min(self._min,other._min)
newset._max = max(self._max,other._max)
newset._ranges = []
for states, (start,stop) in \
self._iterranges(self._ranges,other._ranges,
newset._min,newset._max):
if pall(states):
if newset._ranges and newset._ranges[-1][1] == start:
newset._ranges[-1] = (newset._ranges[-1][0],stop)
else:
newset._ranges.append((start,stop))
newset._ranges = tuple(newset._ranges)
newset._hash = hash(self._ranges)
return newset
elif type == "bool":
def f(self,other):
coerced = self.__coerce__(other)
if coerced is NotImplemented:
return NotImplemented
other = coerced[1]
_min = min(self._min,other._min)
_max = max(self._max,other._max)
found = not pany
for states, (start,stop) in \
self._iterranges(self._ranges,other._ranges,_min,_max):
if not pall(states):
return False
found = found or pany(states)
return found
else:
raise ValueError("Invalid type of function to create.")
try:
f.func_name = name
except TypeError:
pass
f.func_doc = doc
return f
# Intersection.
__and__ = _make_function("__and__","set",
"Intersection of two sets as a new set.",
lambda s: s["r1"] and s["r2"])
__rand__ = _make_function("__rand__","set",
"Intersection of two sets as a new set.",
lambda s: s["r1"] and s["r2"])
intersection = _make_function("intersection","set",
"Intersection of two sets as a new set.",
lambda s: s["r1"] and s["r2"])
# Union.
__or__ = _make_function("__or__","set",
"Union of two sets as a new set.",
lambda s: s["r1"] or s["r2"])
__ror__ = _make_function("__ror__","set",
"Union of two sets as a new set.",
lambda s: s["r1"] or s["r2"])
union = _make_function("union","set",
"Union of two sets as a new set.",
lambda s: s["r1"] or s["r2"])
# Difference.
__sub__ = _make_function("__sub__","set",
"Difference of two sets as a new set.",
lambda s: s["r1"] and not s["r2"])
__rsub__ = _make_function("__rsub__","set",
"Difference of two sets as a new set.",
lambda s: s["r2"] and not s["r1"])
difference = _make_function("difference","set",
"Difference of two sets as a new set.",
lambda s: s["r1"] and not s["r2"])
# Symmetric difference.
__xor__ = _make_function("__xor__","set",
"Symmetric difference of two sets as a new set.",
lambda s: s["r1"] ^ s["r2"])
__rxor__ = _make_function("__rxor__","set",
"Symmetric difference of two sets as a new set.",
lambda s: s["r1"] ^ s["r2"])
symmetric_difference = _make_function("symmetric_difference","set",
"Symmetric difference of two sets as a new set.",
lambda s: s["r1"] ^ s["r2"])
# Containership testing.
__contains__ = _make_function("__contains__","bool",
"Returns true if self is superset of other.",
lambda s: s["r1"] or not s["r2"])
issubset = _make_function("issubset","bool",
"Returns true if self is subset of other.",
lambda s: s["r2"] or not s["r1"])
istruesubset = _make_function("istruesubset","bool",
"Returns true if self is true subset of other.",
lambda s: s["r2"] or not s["r1"],
lambda s: s["r2"] and not s["r1"])
issuperset = _make_function("issuperset","bool",
"Returns true if self is superset of other.",
lambda s: s["r1"] or not s["r2"])
istruesuperset = _make_function("istruesuperset","bool",
"Returns true if self is true superset of other.",
lambda s: s["r1"] or not s["r2"],
lambda s: s["r1"] and not s["r2"])
overlaps = _make_function("overlaps","bool",
"Returns true if self overlaps with other.",
lambda s: True,
lambda s: s["r1"] and s["r2"])
# Comparison.
__eq__ = _make_function("__eq__","bool",
"Returns true if self is equal to other.",
lambda s: not ( s["r1"] ^ s["r2"] ))
__ne__ = _make_function("__ne__","bool",
"Returns true if self is different to other.",
lambda s: True,
lambda s: s["r1"] ^ s["r2"])
# Clean up namespace.
del _make_function
# Define other functions.
def inverse(self):
"""Inverse of set as a new set."""
newset = self.__class__.__new__(self.__class__)
newset._min = self._min
newset._max = self._max
newset._ranges = []
laststop = self._min
for r in self._ranges:
if laststop < r[0]:
newset._ranges.append((laststop,r[0]))
laststop = r[1]
if laststop < self._max:
newset._ranges.append((laststop,self._max))
return newset
__invert__ = inverse
# Hashing
# -------
def __hash__(self):
"""Returns a hash value representing this integer set. As the set is
always stored normalized, the hash value is guaranteed to match for
matching ranges."""
return self._hash
# Iterating
# ---------
def __len__(self):
"""Get length of this integer set. In case the length is larger than
2**31 (including infinitely sized integer sets), it raises an
OverflowError. This is due to len() restricting the size to
0 <= len < 2**31."""
if not self._ranges:
return 0
if self._ranges[0][0] is _MININF or self._ranges[-1][1] is _MAXINF:
raise OverflowError("Infinitely sized integer set.")
rlen = 0
for r in self._ranges:
rlen += r[1]-r[0]
if rlen >= 2**31:
raise OverflowError("Integer set bigger than 2**31.")
return rlen
def len(self):
"""Returns the length of this integer set as an integer. In case the
length is infinite, returns -1. This function exists because of a
limitation of the builtin len() function which expects values in
the range 0 <= len < 2**31. Use this function in case your integer
set might be larger."""
if not self._ranges:
return 0
if self._ranges[0][0] is _MININF or self._ranges[-1][1] is _MAXINF:
return -1
rlen = 0
for r in self._ranges:
rlen += r[1]-r[0]
return rlen
def __nonzero__(self):
"""Returns true if this integer set contains at least one item."""
return bool(self._ranges)
def __iter__(self):
"""Iterate over all values in this integer set. Iteration always starts
by iterating from lowest to highest over the ranges that are bounded.
After processing these, all ranges that are unbounded (maximum 2) are
yielded intermixed."""
ubranges = []
for r in self._ranges:
if r[0] is _MININF:
if r[1] is _MAXINF:
ubranges.extend(([0,1],[-1,-1]))
else:
ubranges.append([r[1]-1,-1])
elif r[1] is _MAXINF:
ubranges.append([r[0],1])
else:
for val in xrange(r[0],r[1]):
yield val
if ubranges:
while True:
for ubrange in ubranges:
yield ubrange[0]
ubrange[0] += ubrange[1]
# Printing
# --------
def __repr__(self):
"""Return a representation of this integer set. The representation is
executable to get an equal integer set."""
rv = []
for start, stop in self._ranges:
if ( isinstance(start,(int,long)) and isinstance(stop,(int,long))
and stop-start == 1 ):
rv.append("%r" % start)
elif isinstance(stop,(int,long)):
rv.append("(%r,%r)" % (start,stop-1))
else:
rv.append("(%r,%r)" % (start,stop))
if self._min is not _MININF:
rv.append("min=%r" % self._min)
if self._max is not _MAXINF:
rv.append("max=%r" % self._max)
return "%s(%s)" % (self.__class__.__name__,",".join(rv))
if __name__ == "__main__":
# Little test script demonstrating functionality.
x = IntSet((10,20),30)
y = IntSet((10,20))
z = IntSet((10,20),30,(15,19),min=0,max=40)
print x
print x&110
print x|110
print x^(15,25)
print x-12
print 12 in x
print x.issubset(x)
print y.issubset(x)
print x.istruesubset(x)
print y.istruesubset(x)
for val in x:
print val
print x.inverse()
print x == z
print x == y
print x <> y
print hash(x)
print hash(z)
print len(x)
print x.len()

View File

@@ -0,0 +1,273 @@
# -*- coding: iso-8859-15 -*-
"""IP4 address range set implementation.
Implements an IPv4-range type.
Copyright (C) 2006, Heiko Wundram.
Released under the MIT-license.
"""
# Version information
# -------------------
__author__ = "Heiko Wundram <me@modelnine.org>"
__version__ = "0.2"
__revision__ = "3"
__date__ = "2006-01-20"
# Imports
# -------
import intset
import socket
# IP4Range class
# --------------
class IP4Range(intset.IntSet):
"""IP4 address range class with efficient storage of address ranges.
Supports all set operations."""
_MINIP4 = 0
_MAXIP4 = (1<<32) - 1
_UNITYTRANS = "".join([chr(n) for n in range(256)])
_IPREMOVE = "0123456789."
def __init__(self,*args):
"""Initialize an ip4range class. The constructor accepts an unlimited
number of arguments that may either be tuples in the form (start,stop),
integers, longs or strings, where start and stop in a tuple may
also be of the form integer, long or string.
Passing an integer or long means passing an IPv4-address that's already
been converted to integer notation, whereas passing a string specifies
an address where this conversion still has to be done. A string
address may be in the following formats:
- 1.2.3.4 - a plain address, interpreted as a single address
- 1.2.3 - a set of addresses, interpreted as 1.2.3.0-1.2.3.255
- localhost - hostname to look up, interpreted as single address
- 1.2.3<->5 - a set of addresses, interpreted as 1.2.3.0-1.2.5.255
- 1.2.0.0/16 - a set of addresses, interpreted as 1.2.0.0-1.2.255.255
Only the first three notations are valid if you use a string address in
a tuple, whereby notation 2 is interpreted as 1.2.3.0 if specified as
lower bound and 1.2.3.255 if specified as upper bound, not as a range
of addresses.
Specifying a range is done with the <-> operator. This is necessary
because '-' might be present in a hostname. '<->' shouldn't be, ever.
"""
# Special case copy constructor.
if len(args) == 1 and isinstance(args[0],IP4Range):
super(IP4Range,self).__init__(args[0])
return
# Convert arguments to tuple syntax.
args = list(args)
for i in range(len(args)):
argval = args[i]
if isinstance(argval,str):
if "<->" in argval:
# Type 4 address.
args[i] = self._parseRange(*argval.split("<->",1))
continue
elif "/" in argval:
# Type 5 address.
args[i] = self._parseMask(*argval.split("/",1))
else:
# Type 1, 2 or 3.
args[i] = self._parseAddrRange(argval)
elif isinstance(argval,tuple):
if len(tuple) <> 2:
raise ValueError("Tuple is of invalid length.")
addr1, addr2 = argval
if isinstance(addr1,str):
addr1 = self._parseAddrRange(addr1)[0]
elif not isinstance(addr1,(int,long)):
raise TypeError("Invalid argument.")
if isinstance(addr2,str):
addr2 = self._parseAddrRange(addr2)[1]
elif not isinstance(addr2,(int,long)):
raise TypeError("Invalid argument.")
args[i] = (addr1,addr2)
elif not isinstance(argval,(int,long)):
raise TypeError("Invalid argument.")
# Initialize the integer set.
super(IP4Range,self).__init__(min=self._MINIP4,max=self._MAXIP4,*args)
# Parsing functions
# -----------------
def _parseRange(self,addr1,addr2):
naddr1, naddr1len = _parseAddr(addr1)
naddr2, naddr2len = _parseAddr(addr2)
if naddr2len < naddr1len:
naddr2 += naddr1&(((1<<((naddr1len-naddr2len)*8))-1)<<
(naddr2len*8))
naddr2len = naddr1len
elif naddr2len > naddr1len:
raise ValueError("Range has more dots than address.")
naddr1 <<= (4-naddr1len)*8
naddr2 <<= (4-naddr2len)*8
naddr2 += (1<<((4-naddr2len)*8))-1
return (naddr1,naddr2)
def _parseMask(self,addr,mask):
naddr, naddrlen = _parseAddr(addr)
naddr <<= (4-naddrlen)*8
try:
if not mask:
masklen = 0
else:
masklen = int(mask)
if not 0 <= masklen <= 32:
raise ValueError
except ValueError:
try:
mask = _parseAddr(mask,False)
except ValueError:
raise ValueError("Mask isn't parseable.")
remaining = 0
masklen = 0
if not mask:
masklen = 0
else:
while not (mask&1):
remaining += 1
while (mask&1):
mask >>= 1
masklen += 1
if remaining+masklen <> 32:
raise ValueError("Mask isn't a proper host mask.")
naddr1 = naddr & (((1<<masklen)-1)<<(32-masklen))
naddr2 = naddr1 + (1<<(32-masklen)) - 1
return (naddr1,naddr2)
def _parseAddrRange(self,addr):
naddr, naddrlen = _parseAddr(addr)
naddr1 = naddr<<((4-naddrlen)*8)
naddr2 = ( (naddr<<((4-naddrlen)*8)) +
(1<<((4-naddrlen)*8)) - 1 )
return (naddr1,naddr2)
# Utility functions
# -----------------
def _int2ip(self,num):
rv = []
for i in range(4):
rv.append(str(num&255))
num >>= 8
return ".".join(reversed(rv))
# Iterating
# ---------
def iteraddresses(self):
"""Returns an iterator which iterates over ips in this iprange. An
IP is returned in string form (e.g. '1.2.3.4')."""
for v in super(IP4Range,self).__iter__():
yield self._int2ip(v)
def iterranges(self):
"""Returns an iterator which iterates over ip-ip ranges which build
this iprange if combined. An ip-ip pair is returned in string form
(e.g. '1.2.3.4-2.3.4.5')."""
for r in self._ranges:
if r[1]-r[0] == 1:
yield self._int2ip(r[0])
else:
yield '%s-%s' % (self._int2ip(r[0]),self._int2ip(r[1]-1))
def itermasks(self):
"""Returns an iterator which iterates over ip/mask pairs which build
this iprange if combined. An IP/Mask pair is returned in string form
(e.g. '1.2.3.0/24')."""
for r in self._ranges:
for v in self._itermasks(r):
yield v
def _itermasks(self,r):
ranges = [r]
while ranges:
cur = ranges.pop()
curmask = 0
while True:
curmasklen = 1<<(32-curmask)
start = (cur[0]+curmasklen-1)&(((1<<curmask)-1)<<(32-curmask))
if start >= cur[0] and start+curmasklen <= cur[1]:
break
else:
curmask += 1
yield "%s/%s" % (self._int2ip(start),curmask)
if cur[0] < start:
ranges.append((cur[0],start))
if cur[1] > start+curmasklen:
ranges.append((start+curmasklen,cur[1]))
__iter__ = iteraddresses
# Printing
# --------
def __repr__(self):
"""Returns a string which can be used to reconstruct this iprange."""
rv = []
for start, stop in self._ranges:
if stop-start == 1:
rv.append("%r" % (self._int2ip(start),))
else:
rv.append("(%r,%r)" % (self._int2ip(start),
self._int2ip(stop-1)))
return "%s(%s)" % (self.__class__.__name__,",".join(rv))
def _parseAddr(addr,lookup=True):
if lookup and addr.translate(IP4Range._UNITYTRANS, IP4Range._IPREMOVE):
try:
addr = socket.gethostbyname(addr)
except socket.error:
raise ValueError("Invalid Hostname as argument.")
naddr = 0
for naddrpos, part in enumerate(addr.split(".")):
if naddrpos >= 4:
raise ValueError("Address contains more than four parts.")
try:
if not part:
part = 0
else:
part = int(part)
if not 0 <= part < 256:
raise ValueError
except ValueError:
raise ValueError("Address part out of range.")
naddr <<= 8
naddr += part
return naddr, naddrpos+1
def ip2int(addr, lookup=True):
return _parseAddr(addr, lookup=lookup)[0]
if __name__ == "__main__":
# Little test script.
x = IP4Range("172.22.162.250/24")
y = IP4Range("172.22.162.250","172.22.163.250","172.22.163.253<->255")
print x
for val in x.itermasks():
print val
for val in y.itermasks():
print val
for val in (x|y).itermasks():
print val
for val in (x^y).iterranges():
print val
for val in x:
print val

View File

@@ -0,0 +1,30 @@
"""
Kill a thread, from http://sebulba.wikispaces.com/recipe+thread2
"""
import types
try:
import ctypes
except ImportError:
raise ImportError(
"You cannot use paste.util.killthread without ctypes installed")
if not hasattr(ctypes, 'pythonapi'):
raise ImportError(
"You cannot use paste.util.killthread without ctypes.pythonapi")
def async_raise(tid, exctype):
"""raises the exception, performs cleanup if needed.
tid is the value given by thread.get_ident() (an integer).
Raise SystemExit to kill a thread."""
if not isinstance(exctype, (types.ClassType, type)):
raise TypeError("Only types can be raised (not instances)")
if not isinstance(tid, int):
raise TypeError("tid must be an integer")
res = ctypes.pythonapi.PyThreadState_SetAsyncExc(ctypes.c_long(tid), ctypes.py_object(exctype))
if res == 0:
raise ValueError("invalid thread id")
elif res != 1:
# """if it returns a number greater than one, you're in trouble,
# and you should call it again with exc=NULL to revert the effect"""
ctypes.pythonapi.PyThreadState_SetAsyncExc(ctypes.c_long(tid), 0)
raise SystemError("PyThreadState_SetAsyncExc failed")

View File

@@ -0,0 +1,152 @@
"""
Helper for looping over sequences, particular in templates.
Often in a loop in a template it's handy to know what's next up,
previously up, if this is the first or last item in the sequence, etc.
These can be awkward to manage in a normal Python loop, but using the
looper you can get a better sense of the context. Use like::
>>> for loop, item in looper(['a', 'b', 'c']):
... print loop.number, item
... if not loop.last:
... print '---'
1 a
---
2 b
---
3 c
"""
__all__ = ['looper']
class looper(object):
"""
Helper for looping (particularly in templates)
Use this like::
for loop, item in looper(seq):
if loop.first:
...
"""
def __init__(self, seq):
self.seq = seq
def __iter__(self):
return looper_iter(self.seq)
def __repr__(self):
return '<%s for %r>' % (
self.__class__.__name__, self.seq)
class looper_iter(object):
def __init__(self, seq):
self.seq = list(seq)
self.pos = 0
def __iter__(self):
return self
def next(self):
if self.pos >= len(self.seq):
raise StopIteration
result = loop_pos(self.seq, self.pos), self.seq[self.pos]
self.pos += 1
return result
class loop_pos(object):
def __init__(self, seq, pos):
self.seq = seq
self.pos = pos
def __repr__(self):
return '<loop pos=%r at %r>' % (
self.seq[pos], pos)
def index(self):
return self.pos
index = property(index)
def number(self):
return self.pos + 1
number = property(number)
def item(self):
return self.seq[self.pos]
item = property(item)
def next(self):
try:
return self.seq[self.pos+1]
except IndexError:
return None
next = property(next)
def previous(self):
if self.pos == 0:
return None
return self.seq[self.pos-1]
previous = property(previous)
def odd(self):
return not self.pos % 2
odd = property(odd)
def even(self):
return self.pos % 2
even = property(even)
def first(self):
return self.pos == 0
first = property(first)
def last(self):
return self.pos == len(self.seq)-1
last = property(last)
def length(self):
return len(self.seq)
length = property(length)
def first_group(self, getter=None):
"""
Returns true if this item is the start of a new group,
where groups mean that some attribute has changed. The getter
can be None (the item itself changes), an attribute name like
``'.attr'``, a function, or a dict key or list index.
"""
if self.first:
return True
return self._compare_group(self.item, self.previous, getter)
def last_group(self, getter=None):
"""
Returns true if this item is the end of a new group,
where groups mean that some attribute has changed. The getter
can be None (the item itself changes), an attribute name like
``'.attr'``, a function, or a dict key or list index.
"""
if self.last:
return True
return self._compare_group(self.item, self.next, getter)
def _compare_group(self, item, other, getter):
if getter is None:
return item != other
elif (isinstance(getter, basestring)
and getter.startswith('.')):
getter = getter[1:]
if getter.endswith('()'):
getter = getter[:-2]
return getattr(item, getter)() != getattr(other, getter)()
else:
return getattr(item, getter) != getattr(other, getter)
elif callable(getter):
return getter(item) != getter(other)
else:
return item[getter] != other[getter]

View File

@@ -0,0 +1,160 @@
"""MIME-Type Parser
This module provides basic functions for handling mime-types. It can handle
matching mime-types against a list of media-ranges. See section 14.1 of
the HTTP specification [RFC 2616] for a complete explanation.
http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.1
Based on mimeparse 0.1.2 by Joe Gregorio:
http://code.google.com/p/mimeparse/
Contents:
- parse_mime_type(): Parses a mime-type into its component parts.
- parse_media_range(): Media-ranges are mime-types with wild-cards and a 'q' quality parameter.
- quality(): Determines the quality ('q') of a mime-type when compared against a list of media-ranges.
- quality_parsed(): Just like quality() except the second parameter must be pre-parsed.
- best_match(): Choose the mime-type with the highest quality ('q') from a list of candidates.
- desired_matches(): Filter against a list of desired mime-types in the order the server prefers.
"""
def parse_mime_type(mime_type):
"""Carves up a mime-type and returns a tuple of the
(type, subtype, params) where 'params' is a dictionary
of all the parameters for the media range.
For example, the media range 'application/xhtml;q=0.5' would
get parsed into:
('application', 'xhtml', {'q', '0.5'})
"""
type = mime_type.split(';')
type, plist = type[0], type[1:]
try:
type, subtype = type.split('/', 1)
except ValueError:
type, subtype = type.strip() or '*', '*'
else:
type = type.strip() or '*'
subtype = subtype.strip() or '*'
params = {}
for param in plist:
param = param.split('=', 1)
if len(param) == 2:
key, value = param[0].strip(), param[1].strip()
if key and value:
params[key] = value
return type, subtype, params
def parse_media_range(range):
"""Carves up a media range and returns a tuple of the
(type, subtype, params) where 'params' is a dictionary
of all the parameters for the media range.
For example, the media range 'application/*;q=0.5' would
get parsed into:
('application', '*', {'q', '0.5'})
In addition this function also guarantees that there
is a value for 'q' in the params dictionary, filling it
in with a proper default if necessary.
"""
type, subtype, params = parse_mime_type(range)
try:
if not 0 <= float(params['q']) <= 1:
raise ValueError
except (KeyError, ValueError):
params['q'] = '1'
return type, subtype, params
def fitness_and_quality_parsed(mime_type, parsed_ranges):
"""Find the best match for a given mime-type against
a list of media_ranges that have already been
parsed by parse_media_range(). Returns a tuple of
the fitness value and the value of the 'q' quality
parameter of the best match, or (-1, 0) if no match
was found. Just as for quality_parsed(), 'parsed_ranges'
must be a list of parsed media ranges."""
best_fitness, best_fit_q = -1, 0
target_type, target_subtype, target_params = parse_media_range(mime_type)
for type, subtype, params in parsed_ranges:
if (type == target_type
or type == '*' or target_type == '*') and (
subtype == target_subtype
or subtype == '*' or target_subtype == '*'):
fitness = 0
if type == target_type:
fitness += 100
if subtype == target_subtype:
fitness += 10
for key in target_params:
if key != 'q' and key in params:
if params[key] == target_params[key]:
fitness += 1
if fitness > best_fitness:
best_fitness = fitness
best_fit_q = params['q']
return best_fitness, float(best_fit_q)
def quality_parsed(mime_type, parsed_ranges):
"""Find the best match for a given mime-type against
a list of media_ranges that have already been
parsed by parse_media_range(). Returns the
'q' quality parameter of the best match, 0 if no
match was found. This function behaves the same as quality()
except that 'parsed_ranges' must be a list of
parsed media ranges."""
return fitness_and_quality_parsed(mime_type, parsed_ranges)[1]
def quality(mime_type, ranges):
"""Returns the quality 'q' of a mime-type when compared
against the media-ranges in ranges. For example:
>>> quality('text/html','text/*;q=0.3, text/html;q=0.7, text/html;level=1, text/html;level=2;q=0.4, */*;q=0.5')
0.7
"""
parsed_ranges = map(parse_media_range, ranges.split(','))
return quality_parsed(mime_type, parsed_ranges)
def best_match(supported, header):
"""Takes a list of supported mime-types and finds the best
match for all the media-ranges listed in header. In case of
ambiguity, whatever comes first in the list will be chosen.
The value of header must be a string that conforms to the format
of the HTTP Accept: header. The value of 'supported' is a list
of mime-types.
>>> best_match(['application/xbel+xml', 'text/xml'], 'text/*;q=0.5,*/*; q=0.1')
'text/xml'
"""
if not supported:
return ''
parsed_header = map(parse_media_range, header.split(','))
best_type = max([
(fitness_and_quality_parsed(mime_type, parsed_header), -n)
for n, mime_type in enumerate(supported)])
return best_type[0][1] and supported[-best_type[1]] or ''
def desired_matches(desired, header):
"""Takes a list of desired mime-types in the order the server prefers to
send them regardless of the browsers preference.
Browsers (such as Firefox) technically want XML over HTML depending on how
one reads the specification. This function is provided for a server to
declare a set of desired mime-types it supports, and returns a subset of
the desired list in the same order should each one be Accepted by the
browser.
>>> desired_matches(['text/html', 'application/xml'], \
... 'text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png')
['text/html', 'application/xml']
>>> desired_matches(['text/html', 'application/xml'], 'application/xml,application/json')
['application/xml']
"""
parsed_ranges = map(parse_media_range, header.split(','))
return [mimetype for mimetype in desired
if quality_parsed(mimetype, parsed_ranges)]

View File

@@ -0,0 +1,397 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
import cgi
import copy
import sys
from UserDict import DictMixin
class MultiDict(DictMixin):
"""
An ordered dictionary that can have multiple values for each key.
Adds the methods getall, getone, mixed, and add to the normal
dictionary interface.
"""
def __init__(self, *args, **kw):
if len(args) > 1:
raise TypeError(
"MultiDict can only be called with one positional argument")
if args:
if hasattr(args[0], 'iteritems'):
items = list(args[0].iteritems())
elif hasattr(args[0], 'items'):
items = args[0].items()
else:
items = list(args[0])
self._items = items
else:
self._items = []
self._items.extend(kw.iteritems())
def __getitem__(self, key):
for k, v in self._items:
if k == key:
return v
raise KeyError(repr(key))
def __setitem__(self, key, value):
try:
del self[key]
except KeyError:
pass
self._items.append((key, value))
def add(self, key, value):
"""
Add the key and value, not overwriting any previous value.
"""
self._items.append((key, value))
def getall(self, key):
"""
Return a list of all values matching the key (may be an empty list)
"""
result = []
for k, v in self._items:
if key == k:
result.append(v)
return result
def getone(self, key):
"""
Get one value matching the key, raising a KeyError if multiple
values were found.
"""
v = self.getall(key)
if not v:
raise KeyError('Key not found: %r' % key)
if len(v) > 1:
raise KeyError('Multiple values match %r: %r' % (key, v))
return v[0]
def mixed(self):
"""
Returns a dictionary where the values are either single
values, or a list of values when a key/value appears more than
once in this dictionary. This is similar to the kind of
dictionary often used to represent the variables in a web
request.
"""
result = {}
multi = {}
for key, value in self._items:
if key in result:
# We do this to not clobber any lists that are
# *actual* values in this dictionary:
if key in multi:
result[key].append(value)
else:
result[key] = [result[key], value]
multi[key] = None
else:
result[key] = value
return result
def dict_of_lists(self):
"""
Returns a dictionary where each key is associated with a
list of values.
"""
result = {}
for key, value in self._items:
if key in result:
result[key].append(value)
else:
result[key] = [value]
return result
def __delitem__(self, key):
items = self._items
found = False
for i in range(len(items)-1, -1, -1):
if items[i][0] == key:
del items[i]
found = True
if not found:
raise KeyError(repr(key))
def __contains__(self, key):
for k, v in self._items:
if k == key:
return True
return False
has_key = __contains__
def clear(self):
self._items = []
def copy(self):
return MultiDict(self)
def setdefault(self, key, default=None):
for k, v in self._items:
if key == k:
return v
self._items.append((key, default))
return default
def pop(self, key, *args):
if len(args) > 1:
raise TypeError, "pop expected at most 2 arguments, got "\
+ repr(1 + len(args))
for i in range(len(self._items)):
if self._items[i][0] == key:
v = self._items[i][1]
del self._items[i]
return v
if args:
return args[0]
else:
raise KeyError(repr(key))
def popitem(self):
return self._items.pop()
def update(self, other=None, **kwargs):
if other is None:
pass
elif hasattr(other, 'items'):
self._items.extend(other.items())
elif hasattr(other, 'keys'):
for k in other.keys():
self._items.append((k, other[k]))
else:
for k, v in other:
self._items.append((k, v))
if kwargs:
self.update(kwargs)
def __repr__(self):
items = ', '.join(['(%r, %r)' % v for v in self._items])
return '%s([%s])' % (self.__class__.__name__, items)
def __len__(self):
return len(self._items)
##
## All the iteration:
##
def keys(self):
return [k for k, v in self._items]
def iterkeys(self):
for k, v in self._items:
yield k
__iter__ = iterkeys
def items(self):
return self._items[:]
def iteritems(self):
return iter(self._items)
def values(self):
return [v for k, v in self._items]
def itervalues(self):
for k, v in self._items:
yield v
class UnicodeMultiDict(DictMixin):
"""
A MultiDict wrapper that decodes returned values to unicode on the
fly. Decoding is not applied to assigned values.
The key/value contents are assumed to be ``str``/``strs`` or
``str``/``FieldStorages`` (as is returned by the ``paste.request.parse_``
functions).
Can optionally also decode keys when the ``decode_keys`` argument is
True.
``FieldStorage`` instances are cloned, and the clone's ``filename``
variable is decoded. Its ``name`` variable is decoded when ``decode_keys``
is enabled.
"""
def __init__(self, multi=None, encoding=None, errors='strict',
decode_keys=False):
self.multi = multi
if encoding is None:
encoding = sys.getdefaultencoding()
self.encoding = encoding
self.errors = errors
self.decode_keys = decode_keys
def _decode_key(self, key):
if self.decode_keys:
try:
key = key.decode(self.encoding, self.errors)
except AttributeError:
pass
return key
def _decode_value(self, value):
"""
Decode the specified value to unicode. Assumes value is a ``str`` or
`FieldStorage`` object.
``FieldStorage`` objects are specially handled.
"""
if isinstance(value, cgi.FieldStorage):
# decode FieldStorage's field name and filename
value = copy.copy(value)
if self.decode_keys:
value.name = value.name.decode(self.encoding, self.errors)
value.filename = value.filename.decode(self.encoding, self.errors)
else:
try:
value = value.decode(self.encoding, self.errors)
except AttributeError:
pass
return value
def __getitem__(self, key):
return self._decode_value(self.multi.__getitem__(key))
def __setitem__(self, key, value):
self.multi.__setitem__(key, value)
def add(self, key, value):
"""
Add the key and value, not overwriting any previous value.
"""
self.multi.add(key, value)
def getall(self, key):
"""
Return a list of all values matching the key (may be an empty list)
"""
return [self._decode_value(v) for v in self.multi.getall(key)]
def getone(self, key):
"""
Get one value matching the key, raising a KeyError if multiple
values were found.
"""
return self._decode_value(self.multi.getone(key))
def mixed(self):
"""
Returns a dictionary where the values are either single
values, or a list of values when a key/value appears more than
once in this dictionary. This is similar to the kind of
dictionary often used to represent the variables in a web
request.
"""
unicode_mixed = {}
for key, value in self.multi.mixed().iteritems():
if isinstance(value, list):
value = [self._decode_value(value) for value in value]
else:
value = self._decode_value(value)
unicode_mixed[self._decode_key(key)] = value
return unicode_mixed
def dict_of_lists(self):
"""
Returns a dictionary where each key is associated with a
list of values.
"""
unicode_dict = {}
for key, value in self.multi.dict_of_lists().iteritems():
value = [self._decode_value(value) for value in value]
unicode_dict[self._decode_key(key)] = value
return unicode_dict
def __delitem__(self, key):
self.multi.__delitem__(key)
def __contains__(self, key):
return self.multi.__contains__(key)
has_key = __contains__
def clear(self):
self.multi.clear()
def copy(self):
return UnicodeMultiDict(self.multi.copy(), self.encoding, self.errors)
def setdefault(self, key, default=None):
return self._decode_value(self.multi.setdefault(key, default))
def pop(self, key, *args):
return self._decode_value(self.multi.pop(key, *args))
def popitem(self):
k, v = self.multi.popitem()
return (self._decode_key(k), self._decode_value(v))
def __repr__(self):
items = ', '.join(['(%r, %r)' % v for v in self.items()])
return '%s([%s])' % (self.__class__.__name__, items)
def __len__(self):
return self.multi.__len__()
##
## All the iteration:
##
def keys(self):
return [self._decode_key(k) for k in self.multi.iterkeys()]
def iterkeys(self):
for k in self.multi.iterkeys():
yield self._decode_key(k)
__iter__ = iterkeys
def items(self):
return [(self._decode_key(k), self._decode_value(v)) for \
k, v in self.multi.iteritems()]
def iteritems(self):
for k, v in self.multi.iteritems():
yield (self._decode_key(k), self._decode_value(v))
def values(self):
return [self._decode_value(v) for v in self.multi.itervalues()]
def itervalues(self):
for v in self.multi.itervalues():
yield self._decode_value(v)
__test__ = {
'general': """
>>> d = MultiDict(a=1, b=2)
>>> d['a']
1
>>> d.getall('c')
[]
>>> d.add('a', 2)
>>> d['a']
1
>>> d.getall('a')
[1, 2]
>>> d['b'] = 4
>>> d.getall('b')
[4]
>>> d.keys()
['a', 'a', 'b']
>>> d.items()
[('a', 1), ('a', 2), ('b', 4)]
>>> d.mixed()
{'a': [1, 2], 'b': 4}
>>> MultiDict([('a', 'b')], c=2)
MultiDict([('a', 'b'), ('c', 2)])
"""}
if __name__ == '__main__':
import doctest
doctest.testmod()

View File

@@ -0,0 +1,98 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
import cgi
import htmlentitydefs
import urllib
import re
__all__ = ['html_quote', 'html_unquote', 'url_quote', 'url_unquote',
'strip_html']
default_encoding = 'UTF-8'
def html_quote(v, encoding=None):
r"""
Quote the value (turned to a string) as HTML. This quotes <, >,
and quotes:
>>> html_quote(1)
'1'
>>> html_quote(None)
''
>>> html_quote('<hey!>')
'&lt;hey!&gt;'
>>> html_quote(u'\u1029')
'\xe1\x80\xa9'
"""
encoding = encoding or default_encoding
if v is None:
return ''
elif isinstance(v, str):
return cgi.escape(v, 1)
elif isinstance(v, unicode):
return cgi.escape(v.encode(encoding), 1)
else:
return cgi.escape(unicode(v).encode(encoding), 1)
_unquote_re = re.compile(r'&([a-zA-Z]+);')
def _entity_subber(match, name2c=htmlentitydefs.name2codepoint):
code = name2c.get(match.group(1))
if code:
return unichr(code)
else:
return match.group(0)
def html_unquote(s, encoding=None):
r"""
Decode the value.
>>> html_unquote('&lt;hey&nbsp;you&gt;')
u'<hey\xa0you>'
>>> html_unquote('')
u''
>>> html_unquote('&blahblah;')
u'&blahblah;'
>>> html_unquote('\xe1\x80\xa9')
u'\u1029'
"""
if isinstance(s, str):
if s == '':
# workaround re.sub('', '', u'') returning '' < 2.5.2
# instead of u'' >= 2.5.2
return u''
s = s.decode(encoding or default_encoding)
return _unquote_re.sub(_entity_subber, s)
def strip_html(s):
# should this use html_unquote?
s = re.sub('<.*?>', '', s)
s = html_unquote(s)
return s
def no_quote(s):
"""
Quoting that doesn't do anything
"""
return s
_comment_quote_re = re.compile(r'\-\s*\>')
# Everything but \r, \n, \t:
_bad_chars_re = re.compile('[\x00-\x08\x0b-\x0c\x0e-\x1f]')
def comment_quote(s):
"""
Quote that makes sure text can't escape a comment
"""
comment = str(s)
#comment = _bad_chars_re.sub('', comment)
#print 'in ', repr(str(s))
#print 'out', repr(comment)
comment = _comment_quote_re.sub('-&gt;', comment)
return comment
url_quote = urllib.quote
url_unquote = urllib.unquote
if __name__ == '__main__':
import doctest
doctest.testmod()

View File

@@ -0,0 +1,171 @@
"""
SCGI-->WSGI application proxy, "SWAP".
(Originally written by Titus Brown.)
This lets an SCGI front-end like mod_scgi be used to execute WSGI
application objects. To use it, subclass the SWAP class like so::
class TestAppHandler(swap.SWAP):
def __init__(self, *args, **kwargs):
self.prefix = '/canal'
self.app_obj = TestAppClass
swap.SWAP.__init__(self, *args, **kwargs)
where 'TestAppClass' is the application object from WSGI and '/canal'
is the prefix for what is served by the SCGI Web-server-side process.
Then execute the SCGI handler "as usual" by doing something like this::
scgi_server.SCGIServer(TestAppHandler, port=4000).serve()
and point mod_scgi (or whatever your SCGI front end is) at port 4000.
Kudos to the WSGI folk for writing a nice PEP & the Quixote folk for
writing a nice extensible SCGI server for Python!
"""
import sys
import time
from scgi import scgi_server
def debug(msg):
timestamp = time.strftime("%Y-%m-%d %H:%M:%S",
time.localtime(time.time()))
sys.stderr.write("[%s] %s\n" % (timestamp, msg))
class SWAP(scgi_server.SCGIHandler):
"""
SCGI->WSGI application proxy: let an SCGI server execute WSGI
application objects.
"""
app_obj = None
prefix = None
def __init__(self, *args, **kwargs):
assert self.app_obj, "must set app_obj"
assert self.prefix is not None, "must set prefix"
args = (self,) + args
scgi_server.SCGIHandler.__init__(*args, **kwargs)
def handle_connection(self, conn):
"""
Handle an individual connection.
"""
input = conn.makefile("r")
output = conn.makefile("w")
environ = self.read_env(input)
environ['wsgi.input'] = input
environ['wsgi.errors'] = sys.stderr
environ['wsgi.version'] = (1, 0)
environ['wsgi.multithread'] = False
environ['wsgi.multiprocess'] = True
environ['wsgi.run_once'] = False
# dunno how SCGI does HTTPS signalling; can't test it myself... @CTB
if environ.get('HTTPS','off') in ('on','1'):
environ['wsgi.url_scheme'] = 'https'
else:
environ['wsgi.url_scheme'] = 'http'
## SCGI does some weird environ manglement. We need to set
## SCRIPT_NAME from 'prefix' and then set PATH_INFO from
## REQUEST_URI.
prefix = self.prefix
path = environ['REQUEST_URI'][len(prefix):].split('?', 1)[0]
environ['SCRIPT_NAME'] = prefix
environ['PATH_INFO'] = path
headers_set = []
headers_sent = []
chunks = []
def write(data):
chunks.append(data)
def start_response(status, response_headers, exc_info=None):
if exc_info:
try:
if headers_sent:
# Re-raise original exception if headers sent
raise exc_info[0], exc_info[1], exc_info[2]
finally:
exc_info = None # avoid dangling circular ref
elif headers_set:
raise AssertionError("Headers already set!")
headers_set[:] = [status, response_headers]
return write
###
result = self.app_obj(environ, start_response)
try:
for data in result:
chunks.append(data)
# Before the first output, send the stored headers
if not headers_set:
# Error -- the app never called start_response
status = '500 Server Error'
response_headers = [('Content-type', 'text/html')]
chunks = ["XXX start_response never called"]
else:
status, response_headers = headers_sent[:] = headers_set
output.write('Status: %s\r\n' % status)
for header in response_headers:
output.write('%s: %s\r\n' % header)
output.write('\r\n')
for data in chunks:
output.write(data)
finally:
if hasattr(result,'close'):
result.close()
# SCGI backends use connection closing to signal 'fini'.
try:
input.close()
output.close()
conn.close()
except IOError, err:
debug("IOError while closing connection ignored: %s" % err)
def serve_application(application, prefix, port=None, host=None, max_children=None):
"""
Serve the specified WSGI application via SCGI proxy.
``application``
The WSGI application to serve.
``prefix``
The prefix for what is served by the SCGI Web-server-side process.
``port``
Optional port to bind the SCGI proxy to. Defaults to SCGIServer's
default port value.
``host``
Optional host to bind the SCGI proxy to. Defaults to SCGIServer's
default host value.
``host``
Optional maximum number of child processes the SCGIServer will
spawn. Defaults to SCGIServer's default max_children value.
"""
class SCGIAppHandler(SWAP):
def __init__ (self, *args, **kwargs):
self.prefix = prefix
self.app_obj = application
SWAP.__init__(self, *args, **kwargs)
kwargs = dict(handler_class=SCGIAppHandler)
for kwarg in ('host', 'port', 'max_children'):
if locals()[kwarg] is not None:
kwargs[kwarg] = locals()[kwarg]
scgi_server.SCGIServer(**kwargs).serve()

View File

@@ -0,0 +1,531 @@
"""A collection of string operations (most are no longer used).
Warning: most of the code you see here isn't normally used nowadays.
Beginning with Python 1.6, many of these functions are implemented as
methods on the standard string object. They used to be implemented by
a built-in module called strop, but strop is now obsolete itself.
Public module variables:
whitespace -- a string containing all characters considered whitespace
lowercase -- a string containing all characters considered lowercase letters
uppercase -- a string containing all characters considered uppercase letters
letters -- a string containing all characters considered letters
digits -- a string containing all characters considered decimal digits
hexdigits -- a string containing all characters considered hexadecimal digits
octdigits -- a string containing all characters considered octal digits
punctuation -- a string containing all characters considered punctuation
printable -- a string containing all characters considered printable
"""
# Some strings for ctype-style character classification
whitespace = ' \t\n\r\v\f'
lowercase = 'abcdefghijklmnopqrstuvwxyz'
uppercase = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ'
letters = lowercase + uppercase
ascii_lowercase = lowercase
ascii_uppercase = uppercase
ascii_letters = ascii_lowercase + ascii_uppercase
digits = '0123456789'
hexdigits = digits + 'abcdef' + 'ABCDEF'
octdigits = '01234567'
punctuation = """!"#$%&'()*+,-./:;<=>?@[\]^_`{|}~"""
printable = digits + letters + punctuation + whitespace
# Case conversion helpers
# Use str to convert Unicode literal in case of -U
# Note that Cookie.py bogusly uses _idmap :(
l = map(chr, xrange(256))
_idmap = str('').join(l)
del l
# Functions which aren't available as string methods.
# Capitalize the words in a string, e.g. " aBc dEf " -> "Abc Def".
# See also regsub.capwords().
def capwords(s, sep=None):
"""capwords(s, [sep]) -> string
Split the argument into words using split, capitalize each
word using capitalize, and join the capitalized words using
join. Note that this replaces runs of whitespace characters by
a single space.
"""
return (sep or ' ').join([x.capitalize() for x in s.split(sep)])
# Construct a translation string
_idmapL = None
def maketrans(fromstr, tostr):
"""maketrans(frm, to) -> string
Return a translation table (a string of 256 bytes long)
suitable for use in string.translate. The strings frm and to
must be of the same length.
"""
if len(fromstr) != len(tostr):
raise ValueError, "maketrans arguments must have same length"
global _idmapL
if not _idmapL:
_idmapL = map(None, _idmap)
L = _idmapL[:]
fromstr = map(ord, fromstr)
for i in range(len(fromstr)):
L[fromstr[i]] = tostr[i]
return ''.join(L)
####################################################################
import re as _re
class _multimap:
"""Helper class for combining multiple mappings.
Used by .{safe_,}substitute() to combine the mapping and keyword
arguments.
"""
def __init__(self, primary, secondary):
self._primary = primary
self._secondary = secondary
def __getitem__(self, key):
try:
return self._primary[key]
except KeyError:
return self._secondary[key]
class _TemplateMetaclass(type):
pattern = r"""
%(delim)s(?:
(?P<escaped>%(delim)s) | # Escape sequence of two delimiters
(?P<named>%(id)s) | # delimiter and a Python identifier
{(?P<braced>%(id)s)} | # delimiter and a braced identifier
(?P<invalid>) # Other ill-formed delimiter exprs
)
"""
def __init__(cls, name, bases, dct):
super(_TemplateMetaclass, cls).__init__(name, bases, dct)
if 'pattern' in dct:
pattern = cls.pattern
else:
pattern = _TemplateMetaclass.pattern % {
'delim' : _re.escape(cls.delimiter),
'id' : cls.idpattern,
}
cls.pattern = _re.compile(pattern, _re.IGNORECASE | _re.VERBOSE)
class Template:
"""A string class for supporting $-substitutions."""
__metaclass__ = _TemplateMetaclass
delimiter = '$'
idpattern = r'[_a-z][_a-z0-9]*'
def __init__(self, template):
self.template = template
# Search for $$, $identifier, ${identifier}, and any bare $'s
def _invalid(self, mo):
i = mo.start('invalid')
lines = self.template[:i].splitlines(True)
if not lines:
colno = 1
lineno = 1
else:
colno = i - len(''.join(lines[:-1]))
lineno = len(lines)
raise ValueError('Invalid placeholder in string: line %d, col %d' %
(lineno, colno))
def substitute(self, *args, **kws):
if len(args) > 1:
raise TypeError('Too many positional arguments')
if not args:
mapping = kws
elif kws:
mapping = _multimap(kws, args[0])
else:
mapping = args[0]
# Helper function for .sub()
def convert(mo):
# Check the most common path first.
named = mo.group('named') or mo.group('braced')
if named is not None:
val = mapping[named]
# We use this idiom instead of str() because the latter will
# fail if val is a Unicode containing non-ASCII characters.
return '%s' % val
if mo.group('escaped') is not None:
return self.delimiter
if mo.group('invalid') is not None:
self._invalid(mo)
raise ValueError('Unrecognized named group in pattern',
self.pattern)
return self.pattern.sub(convert, self.template)
def safe_substitute(self, *args, **kws):
if len(args) > 1:
raise TypeError('Too many positional arguments')
if not args:
mapping = kws
elif kws:
mapping = _multimap(kws, args[0])
else:
mapping = args[0]
# Helper function for .sub()
def convert(mo):
named = mo.group('named')
if named is not None:
try:
# We use this idiom instead of str() because the latter
# will fail if val is a Unicode containing non-ASCII
return '%s' % mapping[named]
except KeyError:
return self.delimiter + named
braced = mo.group('braced')
if braced is not None:
try:
return '%s' % mapping[braced]
except KeyError:
return self.delimiter + '{' + braced + '}'
if mo.group('escaped') is not None:
return self.delimiter
if mo.group('invalid') is not None:
return self.delimiter
raise ValueError('Unrecognized named group in pattern',
self.pattern)
return self.pattern.sub(convert, self.template)
####################################################################
# NOTE: Everything below here is deprecated. Use string methods instead.
# This stuff will go away in Python 3.0.
# Backward compatible names for exceptions
index_error = ValueError
atoi_error = ValueError
atof_error = ValueError
atol_error = ValueError
# convert UPPER CASE letters to lower case
def lower(s):
"""lower(s) -> string
Return a copy of the string s converted to lowercase.
"""
return s.lower()
# Convert lower case letters to UPPER CASE
def upper(s):
"""upper(s) -> string
Return a copy of the string s converted to uppercase.
"""
return s.upper()
# Swap lower case letters and UPPER CASE
def swapcase(s):
"""swapcase(s) -> string
Return a copy of the string s with upper case characters
converted to lowercase and vice versa.
"""
return s.swapcase()
# Strip leading and trailing tabs and spaces
def strip(s, chars=None):
"""strip(s [,chars]) -> string
Return a copy of the string s with leading and trailing
whitespace removed.
If chars is given and not None, remove characters in chars instead.
If chars is unicode, S will be converted to unicode before stripping.
"""
return s.strip(chars)
# Strip leading tabs and spaces
def lstrip(s, chars=None):
"""lstrip(s [,chars]) -> string
Return a copy of the string s with leading whitespace removed.
If chars is given and not None, remove characters in chars instead.
"""
return s.lstrip(chars)
# Strip trailing tabs and spaces
def rstrip(s, chars=None):
"""rstrip(s [,chars]) -> string
Return a copy of the string s with trailing whitespace removed.
If chars is given and not None, remove characters in chars instead.
"""
return s.rstrip(chars)
# Split a string into a list of space/tab-separated words
def split(s, sep=None, maxsplit=-1):
"""split(s [,sep [,maxsplit]]) -> list of strings
Return a list of the words in the string s, using sep as the
delimiter string. If maxsplit is given, splits at no more than
maxsplit places (resulting in at most maxsplit+1 words). If sep
is not specified or is None, any whitespace string is a separator.
(split and splitfields are synonymous)
"""
return s.split(sep, maxsplit)
splitfields = split
# Split a string into a list of space/tab-separated words
def rsplit(s, sep=None, maxsplit=-1):
"""rsplit(s [,sep [,maxsplit]]) -> list of strings
Return a list of the words in the string s, using sep as the
delimiter string, starting at the end of the string and working
to the front. If maxsplit is given, at most maxsplit splits are
done. If sep is not specified or is None, any whitespace string
is a separator.
"""
return s.rsplit(sep, maxsplit)
# Join fields with optional separator
def join(words, sep = ' '):
"""join(list [,sep]) -> string
Return a string composed of the words in list, with
intervening occurrences of sep. The default separator is a
single space.
(joinfields and join are synonymous)
"""
return sep.join(words)
joinfields = join
# Find substring, raise exception if not found
def index(s, *args):
"""index(s, sub [,start [,end]]) -> int
Like find but raises ValueError when the substring is not found.
"""
return s.index(*args)
# Find last substring, raise exception if not found
def rindex(s, *args):
"""rindex(s, sub [,start [,end]]) -> int
Like rfind but raises ValueError when the substring is not found.
"""
return s.rindex(*args)
# Count non-overlapping occurrences of substring
def count(s, *args):
"""count(s, sub[, start[,end]]) -> int
Return the number of occurrences of substring sub in string
s[start:end]. Optional arguments start and end are
interpreted as in slice notation.
"""
return s.count(*args)
# Find substring, return -1 if not found
def find(s, *args):
"""find(s, sub [,start [,end]]) -> in
Return the lowest index in s where substring sub is found,
such that sub is contained within s[start,end]. Optional
arguments start and end are interpreted as in slice notation.
Return -1 on failure.
"""
return s.find(*args)
# Find last substring, return -1 if not found
def rfind(s, *args):
"""rfind(s, sub [,start [,end]]) -> int
Return the highest index in s where substring sub is found,
such that sub is contained within s[start,end]. Optional
arguments start and end are interpreted as in slice notation.
Return -1 on failure.
"""
return s.rfind(*args)
# for a bit of speed
_float = float
_int = int
_long = long
# Convert string to float
def atof(s):
"""atof(s) -> float
Return the floating point number represented by the string s.
"""
return _float(s)
# Convert string to integer
def atoi(s , base=10):
"""atoi(s [,base]) -> int
Return the integer represented by the string s in the given
base, which defaults to 10. The string s must consist of one
or more digits, possibly preceded by a sign. If base is 0, it
is chosen from the leading characters of s, 0 for octal, 0x or
0X for hexadecimal. If base is 16, a preceding 0x or 0X is
accepted.
"""
return _int(s, base)
# Convert string to long integer
def atol(s, base=10):
"""atol(s [,base]) -> long
Return the long integer represented by the string s in the
given base, which defaults to 10. The string s must consist
of one or more digits, possibly preceded by a sign. If base
is 0, it is chosen from the leading characters of s, 0 for
octal, 0x or 0X for hexadecimal. If base is 16, a preceding
0x or 0X is accepted. A trailing L or l is not accepted,
unless base is 0.
"""
return _long(s, base)
# Left-justify a string
def ljust(s, width, *args):
"""ljust(s, width[, fillchar]) -> string
Return a left-justified version of s, in a field of the
specified width, padded with spaces as needed. The string is
never truncated. If specified the fillchar is used instead of spaces.
"""
return s.ljust(width, *args)
# Right-justify a string
def rjust(s, width, *args):
"""rjust(s, width[, fillchar]) -> string
Return a right-justified version of s, in a field of the
specified width, padded with spaces as needed. The string is
never truncated. If specified the fillchar is used instead of spaces.
"""
return s.rjust(width, *args)
# Center a string
def center(s, width, *args):
"""center(s, width[, fillchar]) -> string
Return a center version of s, in a field of the specified
width. padded with spaces as needed. The string is never
truncated. If specified the fillchar is used instead of spaces.
"""
return s.center(width, *args)
# Zero-fill a number, e.g., (12, 3) --> '012' and (-3, 3) --> '-03'
# Decadent feature: the argument may be a string or a number
# (Use of this is deprecated; it should be a string as with ljust c.s.)
def zfill(x, width):
"""zfill(x, width) -> string
Pad a numeric string x with zeros on the left, to fill a field
of the specified width. The string x is never truncated.
"""
if not isinstance(x, basestring):
x = repr(x)
return x.zfill(width)
# Expand tabs in a string.
# Doesn't take non-printing chars into account, but does understand \n.
def expandtabs(s, tabsize=8):
"""expandtabs(s [,tabsize]) -> string
Return a copy of the string s with all tab characters replaced
by the appropriate number of spaces, depending on the current
column, and the tabsize (default 8).
"""
return s.expandtabs(tabsize)
# Character translation through look-up table.
def translate(s, table, deletions=""):
"""translate(s,table [,deletions]) -> string
Return a copy of the string s, where all characters occurring
in the optional argument deletions are removed, and the
remaining characters have been mapped through the given
translation table, which must be a string of length 256. The
deletions argument is not allowed for Unicode strings.
"""
if deletions:
return s.translate(table, deletions)
else:
# Add s[:0] so that if s is Unicode and table is an 8-bit string,
# table is converted to Unicode. This means that table *cannot*
# be a dictionary -- for that feature, use u.translate() directly.
return s.translate(table + s[:0])
# Capitalize a string, e.g. "aBc dEf" -> "Abc def".
def capitalize(s):
"""capitalize(s) -> string
Return a copy of the string s with only its first character
capitalized.
"""
return s.capitalize()
# Substring replacement (global)
def replace(s, old, new, maxsplit=-1):
"""replace (str, old, new[, maxsplit]) -> string
Return a copy of string str with all occurrences of substring
old replaced by new. If the optional argument maxsplit is
given, only the first maxsplit occurrences are replaced.
"""
return s.replace(old, new, maxsplit)
# Try importing optional built-in module "strop" -- if it exists,
# it redefines some string operations that are 100-1000 times faster.
# It also defines values for whitespace, lowercase and uppercase
# that match <ctype.h>'s definitions.
try:
from strop import maketrans, lowercase, uppercase, whitespace
letters = lowercase + uppercase
except ImportError:
pass # Use the original versions

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,758 @@
"""
A small templating language
This implements a small templating language for use internally in
Paste and Paste Script. This language implements if/elif/else,
for/continue/break, expressions, and blocks of Python code. The
syntax is::
{{any expression (function calls etc)}}
{{any expression | filter}}
{{for x in y}}...{{endfor}}
{{if x}}x{{elif y}}y{{else}}z{{endif}}
{{py:x=1}}
{{py:
def foo(bar):
return 'baz'
}}
{{default var = default_value}}
{{# comment}}
You use this with the ``Template`` class or the ``sub`` shortcut.
The ``Template`` class takes the template string and the name of
the template (for errors) and a default namespace. Then (like
``string.Template``) you can call the ``tmpl.substitute(**kw)``
method to make a substitution (or ``tmpl.substitute(a_dict)``).
``sub(content, **kw)`` substitutes the template immediately. You
can use ``__name='tmpl.html'`` to set the name of the template.
If there are syntax errors ``TemplateError`` will be raised.
"""
import re
import sys
import cgi
import urllib
from paste.util.looper import looper
__all__ = ['TemplateError', 'Template', 'sub', 'HTMLTemplate',
'sub_html', 'html', 'bunch']
token_re = re.compile(r'\{\{|\}\}')
in_re = re.compile(r'\s+in\s+')
var_re = re.compile(r'^[a-z_][a-z0-9_]*$', re.I)
class TemplateError(Exception):
"""Exception raised while parsing a template
"""
def __init__(self, message, position, name=None):
self.message = message
self.position = position
self.name = name
def __str__(self):
msg = '%s at line %s column %s' % (
self.message, self.position[0], self.position[1])
if self.name:
msg += ' in %s' % self.name
return msg
class _TemplateContinue(Exception):
pass
class _TemplateBreak(Exception):
pass
class Template(object):
default_namespace = {
'start_braces': '{{',
'end_braces': '}}',
'looper': looper,
}
default_encoding = 'utf8'
def __init__(self, content, name=None, namespace=None):
self.content = content
self._unicode = isinstance(content, unicode)
self.name = name
self._parsed = parse(content, name=name)
if namespace is None:
namespace = {}
self.namespace = namespace
def from_filename(cls, filename, namespace=None, encoding=None):
f = open(filename, 'rb')
c = f.read()
f.close()
if encoding:
c = c.decode(encoding)
return cls(content=c, name=filename, namespace=namespace)
from_filename = classmethod(from_filename)
def __repr__(self):
return '<%s %s name=%r>' % (
self.__class__.__name__,
hex(id(self))[2:], self.name)
def substitute(self, *args, **kw):
if args:
if kw:
raise TypeError(
"You can only give positional *or* keyword arguments")
if len(args) > 1:
raise TypeError(
"You can only give on positional argument")
kw = args[0]
ns = self.default_namespace.copy()
ns.update(self.namespace)
ns.update(kw)
result = self._interpret(ns)
return result
def _interpret(self, ns):
__traceback_hide__ = True
parts = []
self._interpret_codes(self._parsed, ns, out=parts)
return ''.join(parts)
def _interpret_codes(self, codes, ns, out):
__traceback_hide__ = True
for item in codes:
if isinstance(item, basestring):
out.append(item)
else:
self._interpret_code(item, ns, out)
def _interpret_code(self, code, ns, out):
__traceback_hide__ = True
name, pos = code[0], code[1]
if name == 'py':
self._exec(code[2], ns, pos)
elif name == 'continue':
raise _TemplateContinue()
elif name == 'break':
raise _TemplateBreak()
elif name == 'for':
vars, expr, content = code[2], code[3], code[4]
expr = self._eval(expr, ns, pos)
self._interpret_for(vars, expr, content, ns, out)
elif name == 'cond':
parts = code[2:]
self._interpret_if(parts, ns, out)
elif name == 'expr':
parts = code[2].split('|')
base = self._eval(parts[0], ns, pos)
for part in parts[1:]:
func = self._eval(part, ns, pos)
base = func(base)
out.append(self._repr(base, pos))
elif name == 'default':
var, expr = code[2], code[3]
if var not in ns:
result = self._eval(expr, ns, pos)
ns[var] = result
elif name == 'comment':
return
else:
assert 0, "Unknown code: %r" % name
def _interpret_for(self, vars, expr, content, ns, out):
__traceback_hide__ = True
for item in expr:
if len(vars) == 1:
ns[vars[0]] = item
else:
if len(vars) != len(item):
raise ValueError(
'Need %i items to unpack (got %i items)'
% (len(vars), len(item)))
for name, value in zip(vars, item):
ns[name] = value
try:
self._interpret_codes(content, ns, out)
except _TemplateContinue:
continue
except _TemplateBreak:
break
def _interpret_if(self, parts, ns, out):
__traceback_hide__ = True
# @@: if/else/else gets through
for part in parts:
assert not isinstance(part, basestring)
name, pos = part[0], part[1]
if name == 'else':
result = True
else:
result = self._eval(part[2], ns, pos)
if result:
self._interpret_codes(part[3], ns, out)
break
def _eval(self, code, ns, pos):
__traceback_hide__ = True
try:
value = eval(code, ns)
return value
except:
exc_info = sys.exc_info()
e = exc_info[1]
if getattr(e, 'args'):
arg0 = e.args[0]
else:
arg0 = str(e)
e.args = (self._add_line_info(arg0, pos),)
raise exc_info[0], e, exc_info[2]
def _exec(self, code, ns, pos):
__traceback_hide__ = True
try:
exec code in ns
except:
exc_info = sys.exc_info()
e = exc_info[1]
e.args = (self._add_line_info(e.args[0], pos),)
raise exc_info[0], e, exc_info[2]
def _repr(self, value, pos):
__traceback_hide__ = True
try:
if value is None:
return ''
if self._unicode:
try:
value = unicode(value)
except UnicodeDecodeError:
value = str(value)
else:
value = str(value)
except:
exc_info = sys.exc_info()
e = exc_info[1]
e.args = (self._add_line_info(e.args[0], pos),)
raise exc_info[0], e, exc_info[2]
else:
if self._unicode and isinstance(value, str):
if not self.decode_encoding:
raise UnicodeDecodeError(
'Cannot decode str value %r into unicode '
'(no default_encoding provided)' % value)
value = value.decode(self.default_encoding)
elif not self._unicode and isinstance(value, unicode):
if not self.decode_encoding:
raise UnicodeEncodeError(
'Cannot encode unicode value %r into str '
'(no default_encoding provided)' % value)
value = value.encode(self.default_encoding)
return value
def _add_line_info(self, msg, pos):
msg = "%s at line %s column %s" % (
msg, pos[0], pos[1])
if self.name:
msg += " in file %s" % self.name
return msg
def sub(content, **kw):
name = kw.get('__name')
tmpl = Template(content, name=name)
return tmpl.substitute(kw)
return result
def paste_script_template_renderer(content, vars, filename=None):
tmpl = Template(content, name=filename)
return tmpl.substitute(vars)
class bunch(dict):
def __init__(self, **kw):
for name, value in kw.items():
setattr(self, name, value)
def __setattr__(self, name, value):
self[name] = value
def __getattr__(self, name):
try:
return self[name]
except KeyError:
raise AttributeError(name)
def __getitem__(self, key):
if 'default' in self:
try:
return dict.__getitem__(self, key)
except KeyError:
return dict.__getitem__(self, 'default')
else:
return dict.__getitem__(self, key)
def __repr__(self):
items = [
(k, v) for k, v in self.items()]
items.sort()
return '<%s %s>' % (
self.__class__.__name__,
' '.join(['%s=%r' % (k, v) for k, v in items]))
############################################################
## HTML Templating
############################################################
class html(object):
def __init__(self, value):
self.value = value
def __str__(self):
return self.value
def __repr__(self):
return '<%s %r>' % (
self.__class__.__name__, self.value)
def html_quote(value):
if value is None:
return ''
if not isinstance(value, basestring):
if hasattr(value, '__unicode__'):
value = unicode(value)
else:
value = str(value)
value = cgi.escape(value, 1)
if isinstance(value, unicode):
value = value.encode('ascii', 'xmlcharrefreplace')
return value
def url(v):
if not isinstance(v, basestring):
if hasattr(v, '__unicode__'):
v = unicode(v)
else:
v = str(v)
if isinstance(v, unicode):
v = v.encode('utf8')
return urllib.quote(v)
def attr(**kw):
kw = kw.items()
kw.sort()
parts = []
for name, value in kw:
if value is None:
continue
if name.endswith('_'):
name = name[:-1]
parts.append('%s="%s"' % (html_quote(name), html_quote(value)))
return html(' '.join(parts))
class HTMLTemplate(Template):
default_namespace = Template.default_namespace.copy()
default_namespace.update(dict(
html=html,
attr=attr,
url=url,
))
def _repr(self, value, pos):
plain = Template._repr(self, value, pos)
if isinstance(value, html):
return plain
else:
return html_quote(plain)
def sub_html(content, **kw):
name = kw.get('__name')
tmpl = HTMLTemplate(content, name=name)
return tmpl.substitute(kw)
return result
############################################################
## Lexing and Parsing
############################################################
def lex(s, name=None, trim_whitespace=True):
"""
Lex a string into chunks:
>>> lex('hey')
['hey']
>>> lex('hey {{you}}')
['hey ', ('you', (1, 7))]
>>> lex('hey {{')
Traceback (most recent call last):
...
TemplateError: No }} to finish last expression at line 1 column 7
>>> lex('hey }}')
Traceback (most recent call last):
...
TemplateError: }} outside expression at line 1 column 7
>>> lex('hey {{ {{')
Traceback (most recent call last):
...
TemplateError: {{ inside expression at line 1 column 10
"""
in_expr = False
chunks = []
last = 0
last_pos = (1, 1)
for match in token_re.finditer(s):
expr = match.group(0)
pos = find_position(s, match.end())
if expr == '{{' and in_expr:
raise TemplateError('{{ inside expression', position=pos,
name=name)
elif expr == '}}' and not in_expr:
raise TemplateError('}} outside expression', position=pos,
name=name)
if expr == '{{':
part = s[last:match.start()]
if part:
chunks.append(part)
in_expr = True
else:
chunks.append((s[last:match.start()], last_pos))
in_expr = False
last = match.end()
last_pos = pos
if in_expr:
raise TemplateError('No }} to finish last expression',
name=name, position=last_pos)
part = s[last:]
if part:
chunks.append(part)
if trim_whitespace:
chunks = trim_lex(chunks)
return chunks
statement_re = re.compile(r'^(?:if |elif |else |for |py:)')
single_statements = ['endif', 'endfor', 'continue', 'break']
trail_whitespace_re = re.compile(r'\n[\t ]*$')
lead_whitespace_re = re.compile(r'^[\t ]*\n')
def trim_lex(tokens):
r"""
Takes a lexed set of tokens, and removes whitespace when there is
a directive on a line by itself:
>>> tokens = lex('{{if x}}\nx\n{{endif}}\ny', trim_whitespace=False)
>>> tokens
[('if x', (1, 3)), '\nx\n', ('endif', (3, 3)), '\ny']
>>> trim_lex(tokens)
[('if x', (1, 3)), 'x\n', ('endif', (3, 3)), 'y']
"""
for i in range(len(tokens)):
current = tokens[i]
if isinstance(tokens[i], basestring):
# we don't trim this
continue
item = current[0]
if not statement_re.search(item) and item not in single_statements:
continue
if not i:
prev = ''
else:
prev = tokens[i-1]
if i+1 >= len(tokens):
next = ''
else:
next = tokens[i+1]
if (not isinstance(next, basestring)
or not isinstance(prev, basestring)):
continue
if ((not prev or trail_whitespace_re.search(prev))
and (not next or lead_whitespace_re.search(next))):
if prev:
m = trail_whitespace_re.search(prev)
# +1 to leave the leading \n on:
prev = prev[:m.start()+1]
tokens[i-1] = prev
if next:
m = lead_whitespace_re.search(next)
next = next[m.end():]
tokens[i+1] = next
return tokens
def find_position(string, index):
"""Given a string and index, return (line, column)"""
leading = string[:index].splitlines()
return (len(leading), len(leading[-1])+1)
def parse(s, name=None):
r"""
Parses a string into a kind of AST
>>> parse('{{x}}')
[('expr', (1, 3), 'x')]
>>> parse('foo')
['foo']
>>> parse('{{if x}}test{{endif}}')
[('cond', (1, 3), ('if', (1, 3), 'x', ['test']))]
>>> parse('series->{{for x in y}}x={{x}}{{endfor}}')
['series->', ('for', (1, 11), ('x',), 'y', ['x=', ('expr', (1, 27), 'x')])]
>>> parse('{{for x, y in z:}}{{continue}}{{endfor}}')
[('for', (1, 3), ('x', 'y'), 'z', [('continue', (1, 21))])]
>>> parse('{{py:x=1}}')
[('py', (1, 3), 'x=1')]
>>> parse('{{if x}}a{{elif y}}b{{else}}c{{endif}}')
[('cond', (1, 3), ('if', (1, 3), 'x', ['a']), ('elif', (1, 12), 'y', ['b']), ('else', (1, 23), None, ['c']))]
Some exceptions::
>>> parse('{{continue}}')
Traceback (most recent call last):
...
TemplateError: continue outside of for loop at line 1 column 3
>>> parse('{{if x}}foo')
Traceback (most recent call last):
...
TemplateError: No {{endif}} at line 1 column 3
>>> parse('{{else}}')
Traceback (most recent call last):
...
TemplateError: else outside of an if block at line 1 column 3
>>> parse('{{if x}}{{for x in y}}{{endif}}{{endfor}}')
Traceback (most recent call last):
...
TemplateError: Unexpected endif at line 1 column 25
>>> parse('{{if}}{{endif}}')
Traceback (most recent call last):
...
TemplateError: if with no expression at line 1 column 3
>>> parse('{{for x y}}{{endfor}}')
Traceback (most recent call last):
...
TemplateError: Bad for (no "in") in 'x y' at line 1 column 3
>>> parse('{{py:x=1\ny=2}}')
Traceback (most recent call last):
...
TemplateError: Multi-line py blocks must start with a newline at line 1 column 3
"""
tokens = lex(s, name=name)
result = []
while tokens:
next, tokens = parse_expr(tokens, name)
result.append(next)
return result
def parse_expr(tokens, name, context=()):
if isinstance(tokens[0], basestring):
return tokens[0], tokens[1:]
expr, pos = tokens[0]
expr = expr.strip()
if expr.startswith('py:'):
expr = expr[3:].lstrip(' \t')
if expr.startswith('\n'):
expr = expr[1:]
else:
if '\n' in expr:
raise TemplateError(
'Multi-line py blocks must start with a newline',
position=pos, name=name)
return ('py', pos, expr), tokens[1:]
elif expr in ('continue', 'break'):
if 'for' not in context:
raise TemplateError(
'continue outside of for loop',
position=pos, name=name)
return (expr, pos), tokens[1:]
elif expr.startswith('if '):
return parse_cond(tokens, name, context)
elif (expr.startswith('elif ')
or expr == 'else'):
raise TemplateError(
'%s outside of an if block' % expr.split()[0],
position=pos, name=name)
elif expr in ('if', 'elif', 'for'):
raise TemplateError(
'%s with no expression' % expr,
position=pos, name=name)
elif expr in ('endif', 'endfor'):
raise TemplateError(
'Unexpected %s' % expr,
position=pos, name=name)
elif expr.startswith('for '):
return parse_for(tokens, name, context)
elif expr.startswith('default '):
return parse_default(tokens, name, context)
elif expr.startswith('#'):
return ('comment', pos, tokens[0][0]), tokens[1:]
return ('expr', pos, tokens[0][0]), tokens[1:]
def parse_cond(tokens, name, context):
start = tokens[0][1]
pieces = []
context = context + ('if',)
while 1:
if not tokens:
raise TemplateError(
'Missing {{endif}}',
position=start, name=name)
if (isinstance(tokens[0], tuple)
and tokens[0][0] == 'endif'):
return ('cond', start) + tuple(pieces), tokens[1:]
next, tokens = parse_one_cond(tokens, name, context)
pieces.append(next)
def parse_one_cond(tokens, name, context):
(first, pos), tokens = tokens[0], tokens[1:]
content = []
if first.endswith(':'):
first = first[:-1]
if first.startswith('if '):
part = ('if', pos, first[3:].lstrip(), content)
elif first.startswith('elif '):
part = ('elif', pos, first[5:].lstrip(), content)
elif first == 'else':
part = ('else', pos, None, content)
else:
assert 0, "Unexpected token %r at %s" % (first, pos)
while 1:
if not tokens:
raise TemplateError(
'No {{endif}}',
position=pos, name=name)
if (isinstance(tokens[0], tuple)
and (tokens[0][0] == 'endif'
or tokens[0][0].startswith('elif ')
or tokens[0][0] == 'else')):
return part, tokens
next, tokens = parse_expr(tokens, name, context)
content.append(next)
def parse_for(tokens, name, context):
first, pos = tokens[0]
tokens = tokens[1:]
context = ('for',) + context
content = []
assert first.startswith('for ')
if first.endswith(':'):
first = first[:-1]
first = first[3:].strip()
match = in_re.search(first)
if not match:
raise TemplateError(
'Bad for (no "in") in %r' % first,
position=pos, name=name)
vars = first[:match.start()]
if '(' in vars:
raise TemplateError(
'You cannot have () in the variable section of a for loop (%r)'
% vars, position=pos, name=name)
vars = tuple([
v.strip() for v in first[:match.start()].split(',')
if v.strip()])
expr = first[match.end():]
while 1:
if not tokens:
raise TemplateError(
'No {{endfor}}',
position=pos, name=name)
if (isinstance(tokens[0], tuple)
and tokens[0][0] == 'endfor'):
return ('for', pos, vars, expr, content), tokens[1:]
next, tokens = parse_expr(tokens, name, context)
content.append(next)
def parse_default(tokens, name, context):
first, pos = tokens[0]
assert first.startswith('default ')
first = first.split(None, 1)[1]
parts = first.split('=', 1)
if len(parts) == 1:
raise TemplateError(
"Expression must be {{default var=value}}; no = found in %r" % first,
position=pos, name=name)
var = parts[0].strip()
if ',' in var:
raise TemplateError(
"{{default x, y = ...}} is not supported",
position=pos, name=name)
if not var_re.search(var):
raise TemplateError(
"Not a valid variable name for {{default}}: %r"
% var, position=pos, name=name)
expr = parts[1].strip()
return ('default', pos, var, expr), tokens[1:]
_fill_command_usage = """\
%prog [OPTIONS] TEMPLATE arg=value
Use py:arg=value to set a Python value; otherwise all values are
strings.
"""
def fill_command(args=None):
import sys, optparse, pkg_resources, os
if args is None:
args = sys.argv[1:]
dist = pkg_resources.get_distribution('Paste')
parser = optparse.OptionParser(
version=str(dist),
usage=_fill_command_usage)
parser.add_option(
'-o', '--output',
dest='output',
metavar="FILENAME",
help="File to write output to (default stdout)")
parser.add_option(
'--html',
dest='use_html',
action='store_true',
help="Use HTML style filling (including automatic HTML quoting)")
parser.add_option(
'--env',
dest='use_env',
action='store_true',
help="Put the environment in as top-level variables")
options, args = parser.parse_args(args)
if len(args) < 1:
print 'You must give a template filename'
print dir(parser)
assert 0
template_name = args[0]
args = args[1:]
vars = {}
if options.use_env:
vars.update(os.environ)
for value in args:
if '=' not in value:
print 'Bad argument: %r' % value
sys.exit(2)
name, value = value.split('=', 1)
if name.startswith('py:'):
name = name[:3]
value = eval(value)
vars[name] = value
if template_name == '-':
template_content = sys.stdin.read()
template_name = '<stdin>'
else:
f = open(template_name, 'rb')
template_content = f.read()
f.close()
if options.use_html:
TemplateClass = HTMLTemplate
else:
TemplateClass = Template
template = TemplateClass(template_content, name=template_name)
result = template.substitute(vars)
if options.output:
f = open(options.output, 'wb')
f.write(result)
f.close()
else:
sys.stdout.write(result)
if __name__ == '__main__':
from paste.util.template import fill_command
fill_command()

View File

@@ -0,0 +1,250 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
"""
threadedprint.py
================
:author: Ian Bicking
:date: 12 Jul 2004
Multi-threaded printing; allows the output produced via print to be
separated according to the thread.
To use this, you must install the catcher, like::
threadedprint.install()
The installation optionally takes one of three parameters:
default
The default destination for print statements (e.g., ``sys.stdout``).
factory
A function that will produce the stream for a thread, given the
thread's name.
paramwriter
Instead of writing to a file-like stream, this function will be
called like ``paramwriter(thread_name, text)`` for every write.
The thread name is the value returned by
``threading.currentThread().getName()``, a string (typically something
like Thread-N).
You can also submit file-like objects for specific threads, which will
override any of these parameters. To do this, call ``register(stream,
[threadName])``. ``threadName`` is optional, and if not provided the
stream will be registered for the current thread.
If no specific stream is registered for a thread, and no default has
been provided, then an error will occur when anything is written to
``sys.stdout`` (or printed).
Note: the stream's ``write`` method will be called in the thread the
text came from, so you should consider thread safety, especially if
multiple threads share the same writer.
Note: if you want access to the original standard out, use
``sys.__stdout__``.
You may also uninstall this, via::
threadedprint.uninstall()
TODO
----
* Something with ``sys.stderr``.
* Some default handlers. Maybe something that hooks into `logging`.
* Possibly cache the results of ``factory`` calls. This would be a
semantic change.
"""
import threading
import sys
from paste.util import filemixin
class PrintCatcher(filemixin.FileMixin):
def __init__(self, default=None, factory=None, paramwriter=None,
leave_stdout=False):
assert len(filter(lambda x: x is not None,
[default, factory, paramwriter])) <= 1, (
"You can only provide one of default, factory, or paramwriter")
if leave_stdout:
assert not default, (
"You cannot pass in both default (%r) and "
"leave_stdout=True" % default)
default = sys.stdout
if default:
self._defaultfunc = self._writedefault
elif factory:
self._defaultfunc = self._writefactory
elif paramwriter:
self._defaultfunc = self._writeparam
else:
self._defaultfunc = self._writeerror
self._default = default
self._factory = factory
self._paramwriter = paramwriter
self._catchers = {}
def write(self, v, currentThread=threading.currentThread):
name = currentThread().getName()
catchers = self._catchers
if not catchers.has_key(name):
self._defaultfunc(name, v)
else:
catcher = catchers[name]
catcher.write(v)
def seek(self, *args):
# Weird, but Google App Engine is seeking on stdout
name = threading.currentThread().getName()
catchers = self._catchers
if not name in catchers:
self._default.seek(*args)
else:
catchers[name].seek(*args)
def read(self, *args):
name = threading.currentThread().getName()
catchers = self._catchers
if not name in catchers:
self._default.read(*args)
else:
catchers[name].read(*args)
def _writedefault(self, name, v):
self._default.write(v)
def _writefactory(self, name, v):
self._factory(name).write(v)
def _writeparam(self, name, v):
self._paramwriter(name, v)
def _writeerror(self, name, v):
assert False, (
"There is no PrintCatcher output stream for the thread %r"
% name)
def register(self, catcher, name=None,
currentThread=threading.currentThread):
if name is None:
name = currentThread().getName()
self._catchers[name] = catcher
def deregister(self, name=None,
currentThread=threading.currentThread):
if name is None:
name = currentThread().getName()
assert self._catchers.has_key(name), (
"There is no PrintCatcher catcher for the thread %r" % name)
del self._catchers[name]
_printcatcher = None
_oldstdout = None
def install(**kw):
global _printcatcher, _oldstdout, register, deregister
if (not _printcatcher or sys.stdout is not _printcatcher):
_oldstdout = sys.stdout
_printcatcher = sys.stdout = PrintCatcher(**kw)
register = _printcatcher.register
deregister = _printcatcher.deregister
def uninstall():
global _printcatcher, _oldstdout, register, deregister
if _printcatcher:
sys.stdout = _oldstdout
_printcatcher = _oldstdout = None
register = not_installed_error
deregister = not_installed_error
def not_installed_error(*args, **kw):
assert False, (
"threadedprint has not yet been installed (call "
"threadedprint.install())")
register = deregister = not_installed_error
class StdinCatcher(filemixin.FileMixin):
def __init__(self, default=None, factory=None, paramwriter=None):
assert len(filter(lambda x: x is not None,
[default, factory, paramwriter])) <= 1, (
"You can only provide one of default, factory, or paramwriter")
if default:
self._defaultfunc = self._readdefault
elif factory:
self._defaultfunc = self._readfactory
elif paramwriter:
self._defaultfunc = self._readparam
else:
self._defaultfunc = self._readerror
self._default = default
self._factory = factory
self._paramwriter = paramwriter
self._catchers = {}
def read(self, size=None, currentThread=threading.currentThread):
name = currentThread().getName()
catchers = self._catchers
if not catchers.has_key(name):
return self._defaultfunc(name, size)
else:
catcher = catchers[name]
return catcher.read(size)
def _readdefault(self, name, size):
self._default.read(size)
def _readfactory(self, name, size):
self._factory(name).read(size)
def _readparam(self, name, size):
self._paramreader(name, size)
def _readerror(self, name, size):
assert False, (
"There is no StdinCatcher output stream for the thread %r"
% name)
def register(self, catcher, name=None,
currentThread=threading.currentThread):
if name is None:
name = currentThread().getName()
self._catchers[name] = catcher
def deregister(self, catcher, name=None,
currentThread=threading.currentThread):
if name is None:
name = currentThread().getName()
assert self._catchers.has_key(name), (
"There is no StdinCatcher catcher for the thread %r" % name)
del self._catchers[name]
_stdincatcher = None
_oldstdin = None
def install_stdin(**kw):
global _stdincatcher, _oldstdin, register_stdin, deregister_stdin
if not _stdincatcher:
_oldstdin = sys.stdin
_stdincatcher = sys.stdin = StdinCatcher(**kw)
register_stdin = _stdincatcher.register
deregister_stdin = _stdincatcher.deregister
def uninstall():
global _stdincatcher, _oldstin, register_stdin, deregister_stdin
if _stdincatcher:
sys.stdin = _oldstdin
_stdincatcher = _oldstdin = None
register_stdin = deregister_stdin = not_installed_error_stdin
def not_installed_error_stdin(*args, **kw):
assert False, (
"threadedprint has not yet been installed for stdin (call "
"threadedprint.install_stdin())")

View File

@@ -0,0 +1,43 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
"""
Implementation of thread-local storage, for Python versions that don't
have thread local storage natively.
"""
try:
import threading
except ImportError:
# No threads, so "thread local" means process-global
class local(object):
pass
else:
try:
local = threading.local
except AttributeError:
# Added in 2.4, but now we'll have to define it ourselves
import thread
class local(object):
def __init__(self):
self.__dict__['__objs'] = {}
def __getattr__(self, attr, g=thread.get_ident):
try:
return self.__dict__['__objs'][g()][attr]
except KeyError:
raise AttributeError(
"No variable %s defined for the thread %s"
% (attr, g()))
def __setattr__(self, attr, value, g=thread.get_ident):
self.__dict__['__objs'].setdefault(g(), {})[attr] = value
def __delattr__(self, attr, g=thread.get_ident):
try:
del self.__dict__['__objs'][g()][attr]
except KeyError:
raise AttributeError(
"No variable %s defined for thread %s"
% (attr, g()))

View File

@@ -0,0 +1,597 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
"""
A module of many disparate routines.
"""
# functions which moved to paste.request and paste.response
# Deprecated around 15 Dec 2005
from paste.request import get_cookies, parse_querystring, parse_formvars
from paste.request import construct_url, path_info_split, path_info_pop
from paste.response import HeaderDict, has_header, header_value, remove_header
from paste.response import error_body_response, error_response, error_response_app
from traceback import print_exception
import urllib
from cStringIO import StringIO
import sys
from urlparse import urlsplit
import warnings
__all__ = ['add_close', 'add_start_close', 'capture_output', 'catch_errors',
'catch_errors_app', 'chained_app_iters', 'construct_url',
'dump_environ', 'encode_unicode_app_iter', 'error_body_response',
'error_response', 'get_cookies', 'has_header', 'header_value',
'interactive', 'intercept_output', 'path_info_pop',
'path_info_split', 'raw_interactive', 'send_file']
class add_close(object):
"""
An an iterable that iterates over app_iter, then calls
close_func.
"""
def __init__(self, app_iterable, close_func):
self.app_iterable = app_iterable
self.app_iter = iter(app_iterable)
self.close_func = close_func
self._closed = False
def __iter__(self):
return self
def next(self):
return self.app_iter.next()
def close(self):
self._closed = True
if hasattr(self.app_iterable, 'close'):
self.app_iterable.close()
self.close_func()
def __del__(self):
if not self._closed:
# We can't raise an error or anything at this stage
print >> sys.stderr, (
"Error: app_iter.close() was not called when finishing "
"WSGI request. finalization function %s not called"
% self.close_func)
class add_start_close(object):
"""
An an iterable that iterates over app_iter, calls start_func
before the first item is returned, then calls close_func at the
end.
"""
def __init__(self, app_iterable, start_func, close_func=None):
self.app_iterable = app_iterable
self.app_iter = iter(app_iterable)
self.first = True
self.start_func = start_func
self.close_func = close_func
self._closed = False
def __iter__(self):
return self
def next(self):
if self.first:
self.start_func()
self.first = False
return self.app_iter.next()
def close(self):
self._closed = True
if hasattr(self.app_iterable, 'close'):
self.app_iterable.close()
if self.close_func is not None:
self.close_func()
def __del__(self):
if not self._closed:
# We can't raise an error or anything at this stage
print >> sys.stderr, (
"Error: app_iter.close() was not called when finishing "
"WSGI request. finalization function %s not called"
% self.close_func)
class chained_app_iters(object):
"""
Chains several app_iters together, also delegating .close() to each
of them.
"""
def __init__(self, *chained):
self.app_iters = chained
self.chained = [iter(item) for item in chained]
self._closed = False
def __iter__(self):
return self
def next(self):
if len(self.chained) == 1:
return self.chained[0].next()
else:
try:
return self.chained[0].next()
except StopIteration:
self.chained.pop(0)
return self.next()
def close(self):
self._closed = True
got_exc = None
for app_iter in self.app_iters:
try:
if hasattr(app_iter, 'close'):
app_iter.close()
except:
got_exc = sys.exc_info()
if got_exc:
raise got_exc[0], got_exc[1], got_exc[2]
def __del__(self):
if not self._closed:
# We can't raise an error or anything at this stage
print >> sys.stderr, (
"Error: app_iter.close() was not called when finishing "
"WSGI request. finalization function %s not called"
% self.close_func)
class encode_unicode_app_iter(object):
"""
Encodes an app_iterable's unicode responses as strings
"""
def __init__(self, app_iterable, encoding=sys.getdefaultencoding(),
errors='strict'):
self.app_iterable = app_iterable
self.app_iter = iter(app_iterable)
self.encoding = encoding
self.errors = errors
def __iter__(self):
return self
def next(self):
content = self.app_iter.next()
if isinstance(content, unicode):
content = content.encode(self.encoding, self.errors)
return content
def close(self):
if hasattr(self.app_iterable, 'close'):
self.app_iterable.close()
def catch_errors(application, environ, start_response, error_callback,
ok_callback=None):
"""
Runs the application, and returns the application iterator (which should be
passed upstream). If an error occurs then error_callback will be called with
exc_info as its sole argument. If no errors occur and ok_callback is given,
then it will be called with no arguments.
"""
try:
app_iter = application(environ, start_response)
except:
error_callback(sys.exc_info())
raise
if type(app_iter) in (list, tuple):
# These won't produce exceptions
if ok_callback:
ok_callback()
return app_iter
else:
return _wrap_app_iter(app_iter, error_callback, ok_callback)
class _wrap_app_iter(object):
def __init__(self, app_iterable, error_callback, ok_callback):
self.app_iterable = app_iterable
self.app_iter = iter(app_iterable)
self.error_callback = error_callback
self.ok_callback = ok_callback
if hasattr(self.app_iterable, 'close'):
self.close = self.app_iterable.close
def __iter__(self):
return self
def next(self):
try:
return self.app_iter.next()
except StopIteration:
if self.ok_callback:
self.ok_callback()
raise
except:
self.error_callback(sys.exc_info())
raise
def catch_errors_app(application, environ, start_response, error_callback_app,
ok_callback=None, catch=Exception):
"""
Like ``catch_errors``, except error_callback_app should be a
callable that will receive *three* arguments -- ``environ``,
``start_response``, and ``exc_info``. It should call
``start_response`` (*with* the exc_info argument!) and return an
iterator.
"""
try:
app_iter = application(environ, start_response)
except catch:
return error_callback_app(environ, start_response, sys.exc_info())
if type(app_iter) in (list, tuple):
# These won't produce exceptions
if ok_callback is not None:
ok_callback()
return app_iter
else:
return _wrap_app_iter_app(
environ, start_response, app_iter,
error_callback_app, ok_callback, catch=catch)
class _wrap_app_iter_app(object):
def __init__(self, environ, start_response, app_iterable,
error_callback_app, ok_callback, catch=Exception):
self.environ = environ
self.start_response = start_response
self.app_iterable = app_iterable
self.app_iter = iter(app_iterable)
self.error_callback_app = error_callback_app
self.ok_callback = ok_callback
self.catch = catch
if hasattr(self.app_iterable, 'close'):
self.close = self.app_iterable.close
def __iter__(self):
return self
def next(self):
try:
return self.app_iter.next()
except StopIteration:
if self.ok_callback:
self.ok_callback()
raise
except self.catch:
if hasattr(self.app_iterable, 'close'):
try:
self.app_iterable.close()
except:
# @@: Print to wsgi.errors?
pass
new_app_iterable = self.error_callback_app(
self.environ, self.start_response, sys.exc_info())
app_iter = iter(new_app_iterable)
if hasattr(new_app_iterable, 'close'):
self.close = new_app_iterable.close
self.next = app_iter.next
return self.next()
def raw_interactive(application, path='', raise_on_wsgi_error=False,
**environ):
"""
Runs the application in a fake environment.
"""
assert "path_info" not in environ, "argument list changed"
if raise_on_wsgi_error:
errors = ErrorRaiser()
else:
errors = StringIO()
basic_environ = {
# mandatory CGI variables
'REQUEST_METHOD': 'GET', # always mandatory
'SCRIPT_NAME': '', # may be empty if app is at the root
'PATH_INFO': '', # may be empty if at root of app
'SERVER_NAME': 'localhost', # always mandatory
'SERVER_PORT': '80', # always mandatory
'SERVER_PROTOCOL': 'HTTP/1.0',
# mandatory wsgi variables
'wsgi.version': (1, 0),
'wsgi.url_scheme': 'http',
'wsgi.input': StringIO(''),
'wsgi.errors': errors,
'wsgi.multithread': False,
'wsgi.multiprocess': False,
'wsgi.run_once': False,
}
if path:
(_, _, path_info, query, fragment) = urlsplit(str(path))
path_info = urllib.unquote(path_info)
# urlsplit returns unicode so coerce it back to str
path_info, query = str(path_info), str(query)
basic_environ['PATH_INFO'] = path_info
if query:
basic_environ['QUERY_STRING'] = query
for name, value in environ.items():
name = name.replace('__', '.')
basic_environ[name] = value
if ('SERVER_NAME' in basic_environ
and 'HTTP_HOST' not in basic_environ):
basic_environ['HTTP_HOST'] = basic_environ['SERVER_NAME']
istream = basic_environ['wsgi.input']
if isinstance(istream, str):
basic_environ['wsgi.input'] = StringIO(istream)
basic_environ['CONTENT_LENGTH'] = len(istream)
data = {}
output = []
headers_set = []
headers_sent = []
def start_response(status, headers, exc_info=None):
if exc_info:
try:
if headers_sent:
# Re-raise original exception only if headers sent
raise exc_info[0], exc_info[1], exc_info[2]
finally:
# avoid dangling circular reference
exc_info = None
elif headers_set:
# You cannot set the headers more than once, unless the
# exc_info is provided.
raise AssertionError("Headers already set and no exc_info!")
headers_set.append(True)
data['status'] = status
data['headers'] = headers
return output.append
app_iter = application(basic_environ, start_response)
try:
try:
for s in app_iter:
if not isinstance(s, str):
raise ValueError(
"The app_iter response can only contain str (not "
"unicode); got: %r" % s)
headers_sent.append(True)
if not headers_set:
raise AssertionError("Content sent w/o headers!")
output.append(s)
except TypeError, e:
# Typically "iteration over non-sequence", so we want
# to give better debugging information...
e.args = ((e.args[0] + ' iterable: %r' % app_iter),) + e.args[1:]
raise
finally:
if hasattr(app_iter, 'close'):
app_iter.close()
return (data['status'], data['headers'], ''.join(output),
errors.getvalue())
class ErrorRaiser(object):
def flush(self):
pass
def write(self, value):
if not value:
return
raise AssertionError(
"No errors should be written (got: %r)" % value)
def writelines(self, seq):
raise AssertionError(
"No errors should be written (got lines: %s)" % list(seq))
def getvalue(self):
return ''
def interactive(*args, **kw):
"""
Runs the application interatively, wrapping `raw_interactive` but
returning the output in a formatted way.
"""
status, headers, content, errors = raw_interactive(*args, **kw)
full = StringIO()
if errors:
full.write('Errors:\n')
full.write(errors.strip())
full.write('\n----------end errors\n')
full.write(status + '\n')
for name, value in headers:
full.write('%s: %s\n' % (name, value))
full.write('\n')
full.write(content)
return full.getvalue()
interactive.proxy = 'raw_interactive'
def dump_environ(environ, start_response):
"""
Application which simply dumps the current environment
variables out as a plain text response.
"""
output = []
keys = environ.keys()
keys.sort()
for k in keys:
v = str(environ[k]).replace("\n","\n ")
output.append("%s: %s\n" % (k, v))
output.append("\n")
content_length = environ.get("CONTENT_LENGTH", '')
if content_length:
output.append(environ['wsgi.input'].read(int(content_length)))
output.append("\n")
output = "".join(output)
headers = [('Content-Type', 'text/plain'),
('Content-Length', str(len(output)))]
start_response("200 OK", headers)
return [output]
def send_file(filename):
warnings.warn(
"wsgilib.send_file has been moved to paste.fileapp.FileApp",
DeprecationWarning, 2)
from paste import fileapp
return fileapp.FileApp(filename)
def capture_output(environ, start_response, application):
"""
Runs application with environ and start_response, and captures
status, headers, and body.
Sends status and header, but *not* body. Returns (status,
headers, body). Typically this is used like:
.. code-block:: python
def dehtmlifying_middleware(application):
def replacement_app(environ, start_response):
status, headers, body = capture_output(
environ, start_response, application)
content_type = header_value(headers, 'content-type')
if (not content_type
or not content_type.startswith('text/html')):
return [body]
body = re.sub(r'<.*?>', '', body)
return [body]
return replacement_app
"""
warnings.warn(
'wsgilib.capture_output has been deprecated in favor '
'of wsgilib.intercept_output',
DeprecationWarning, 2)
data = []
output = StringIO()
def replacement_start_response(status, headers, exc_info=None):
if data:
data[:] = []
data.append(status)
data.append(headers)
start_response(status, headers, exc_info)
return output.write
app_iter = application(environ, replacement_start_response)
try:
for item in app_iter:
output.write(item)
finally:
if hasattr(app_iter, 'close'):
app_iter.close()
if not data:
data.append(None)
if len(data) < 2:
data.append(None)
data.append(output.getvalue())
return data
def intercept_output(environ, application, conditional=None,
start_response=None):
"""
Runs application with environ and captures status, headers, and
body. None are sent on; you must send them on yourself (unlike
``capture_output``)
Typically this is used like:
.. code-block:: python
def dehtmlifying_middleware(application):
def replacement_app(environ, start_response):
status, headers, body = intercept_output(
environ, application)
start_response(status, headers)
content_type = header_value(headers, 'content-type')
if (not content_type
or not content_type.startswith('text/html')):
return [body]
body = re.sub(r'<.*?>', '', body)
return [body]
return replacement_app
A third optional argument ``conditional`` should be a function
that takes ``conditional(status, headers)`` and returns False if
the request should not be intercepted. In that case
``start_response`` will be called and ``(None, None, app_iter)``
will be returned. You must detect that in your code and return
the app_iter, like:
.. code-block:: python
def dehtmlifying_middleware(application):
def replacement_app(environ, start_response):
status, headers, body = intercept_output(
environ, application,
lambda s, h: header_value(headers, 'content-type').startswith('text/html'),
start_response)
if status is None:
return body
start_response(status, headers)
body = re.sub(r'<.*?>', '', body)
return [body]
return replacement_app
"""
if conditional is not None and start_response is None:
raise TypeError(
"If you provide conditional you must also provide "
"start_response")
data = []
output = StringIO()
def replacement_start_response(status, headers, exc_info=None):
if conditional is not None and not conditional(status, headers):
data.append(None)
return start_response(status, headers, exc_info)
if data:
data[:] = []
data.append(status)
data.append(headers)
return output.write
app_iter = application(environ, replacement_start_response)
if data[0] is None:
return (None, None, app_iter)
try:
for item in app_iter:
output.write(item)
finally:
if hasattr(app_iter, 'close'):
app_iter.close()
if not data:
data.append(None)
if len(data) < 2:
data.append(None)
data.append(output.getvalue())
return data
## Deprecation warning wrapper:
class ResponseHeaderDict(HeaderDict):
def __init__(self, *args, **kw):
warnings.warn(
"The class wsgilib.ResponseHeaderDict has been moved "
"to paste.response.HeaderDict",
DeprecationWarning, 2)
HeaderDict.__init__(self, *args, **kw)
def _warn_deprecated(new_func):
new_name = new_func.func_name
new_path = new_func.func_globals['__name__'] + '.' + new_name
def replacement(*args, **kw):
warnings.warn(
"The function wsgilib.%s has been moved to %s"
% (new_name, new_path),
DeprecationWarning, 2)
return new_func(*args, **kw)
try:
replacement.func_name = new_func.func_name
except:
pass
return replacement
# Put warnings wrapper in place for all public functions that
# were imported from elsewhere:
for _name in __all__:
_func = globals()[_name]
if (hasattr(_func, 'func_globals')
and _func.func_globals['__name__'] != __name__):
globals()[_name] = _warn_deprecated(_func)
if __name__ == '__main__':
import doctest
doctest.testmod()

View File

@@ -0,0 +1,581 @@
# (c) 2005 Ian Bicking and contributors; written for Paste (http://pythonpaste.org)
# Licensed under the MIT license: http://www.opensource.org/licenses/mit-license.php
"""WSGI Wrappers for a Request and Response
The WSGIRequest and WSGIResponse objects are light wrappers to make it easier
to deal with an incoming request and sending a response.
"""
import re
import warnings
from pprint import pformat
from Cookie import SimpleCookie
from paste.request import EnvironHeaders, get_cookie_dict, \
parse_dict_querystring, parse_formvars
from paste.util.multidict import MultiDict, UnicodeMultiDict
from paste.registry import StackedObjectProxy
from paste.response import HeaderDict
from paste.wsgilib import encode_unicode_app_iter
from paste.httpheaders import ACCEPT_LANGUAGE
from paste.util.mimeparse import desired_matches
__all__ = ['WSGIRequest', 'WSGIResponse']
_CHARSET_RE = re.compile(r';\s*charset=([^;]*)', re.I)
class DeprecatedSettings(StackedObjectProxy):
def _push_object(self, obj):
warnings.warn('paste.wsgiwrappers.settings is deprecated: Please use '
'paste.wsgiwrappers.WSGIRequest.defaults instead',
DeprecationWarning, 3)
WSGIResponse.defaults._push_object(obj)
StackedObjectProxy._push_object(self, obj)
# settings is deprecated: use WSGIResponse.defaults instead
settings = DeprecatedSettings(default=dict())
class environ_getter(object):
"""For delegating an attribute to a key in self.environ."""
# @@: Also __set__? Should setting be allowed?
def __init__(self, key, default='', default_factory=None):
self.key = key
self.default = default
self.default_factory = default_factory
def __get__(self, obj, type=None):
if type is None:
return self
if self.key not in obj.environ:
if self.default_factory:
val = obj.environ[self.key] = self.default_factory()
return val
else:
return self.default
return obj.environ[self.key]
def __repr__(self):
return '<Proxy for WSGI environ %r key>' % self.key
class WSGIRequest(object):
"""WSGI Request API Object
This object represents a WSGI request with a more friendly interface.
This does not expose every detail of the WSGI environment, and attempts
to express nothing beyond what is available in the environment
dictionary.
The only state maintained in this object is the desired ``charset``,
its associated ``errors`` handler, and the ``decode_param_names``
option.
The incoming parameter values will be automatically coerced to unicode
objects of the ``charset`` encoding when ``charset`` is set. The
incoming parameter names are not decoded to unicode unless the
``decode_param_names`` option is enabled.
When unicode is expected, ``charset`` will overridden by the the
value of the ``Content-Type`` header's charset parameter if one was
specified by the client.
The class variable ``defaults`` specifies default values for
``charset``, ``errors``, and ``langauge``. These can be overridden for the
current request via the registry.
The ``language`` default value is considered the fallback during i18n
translations to ensure in odd cases that mixed languages don't occur should
the ``language`` file contain the string but not another language in the
accepted languages list. The ``language`` value only applies when getting
a list of accepted languages from the HTTP Accept header.
This behavior is duplicated from Aquarium, and may seem strange but is
very useful. Normally, everything in the code is in "en-us". However,
the "en-us" translation catalog is usually empty. If the user requests
``["en-us", "zh-cn"]`` and a translation isn't found for a string in
"en-us", you don't want gettext to fallback to "zh-cn". You want it to
just use the string itself. Hence, if a string isn't found in the
``language`` catalog, the string in the source code will be used.
*All* other state is kept in the environment dictionary; this is
essential for interoperability.
You are free to subclass this object.
"""
defaults = StackedObjectProxy(default=dict(charset=None, errors='replace',
decode_param_names=False,
language='en-us'))
def __init__(self, environ):
self.environ = environ
# This isn't "state" really, since the object is derivative:
self.headers = EnvironHeaders(environ)
defaults = self.defaults._current_obj()
self.charset = defaults.get('charset')
if self.charset:
# There's a charset: params will be coerced to unicode. In that
# case, attempt to use the charset specified by the browser
browser_charset = self.determine_browser_charset()
if browser_charset:
self.charset = browser_charset
self.errors = defaults.get('errors', 'strict')
self.decode_param_names = defaults.get('decode_param_names', False)
self._languages = None
body = environ_getter('wsgi.input')
scheme = environ_getter('wsgi.url_scheme')
method = environ_getter('REQUEST_METHOD')
script_name = environ_getter('SCRIPT_NAME')
path_info = environ_getter('PATH_INFO')
def urlvars(self):
"""
Return any variables matched in the URL (e.g.,
``wsgiorg.routing_args``).
"""
if 'paste.urlvars' in self.environ:
return self.environ['paste.urlvars']
elif 'wsgiorg.routing_args' in self.environ:
return self.environ['wsgiorg.routing_args'][1]
else:
return {}
urlvars = property(urlvars, doc=urlvars.__doc__)
def is_xhr(self):
"""Returns a boolean if X-Requested-With is present and a XMLHttpRequest"""
return self.environ.get('HTTP_X_REQUESTED_WITH', '') == 'XMLHttpRequest'
is_xhr = property(is_xhr, doc=is_xhr.__doc__)
def host(self):
"""Host name provided in HTTP_HOST, with fall-back to SERVER_NAME"""
return self.environ.get('HTTP_HOST', self.environ.get('SERVER_NAME'))
host = property(host, doc=host.__doc__)
def languages(self):
"""Return a list of preferred languages, most preferred first.
The list may be empty.
"""
if self._languages is not None:
return self._languages
acceptLanguage = self.environ.get('HTTP_ACCEPT_LANGUAGE')
langs = ACCEPT_LANGUAGE.parse(self.environ)
fallback = self.defaults.get('language', 'en-us')
if not fallback:
return langs
if fallback not in langs:
langs.append(fallback)
index = langs.index(fallback)
langs[index+1:] = []
self._languages = langs
return self._languages
languages = property(languages, doc=languages.__doc__)
def _GET(self):
return parse_dict_querystring(self.environ)
def GET(self):
"""
Dictionary-like object representing the QUERY_STRING
parameters. Always present, if possibly empty.
If the same key is present in the query string multiple times, a
list of its values can be retrieved from the ``MultiDict`` via
the ``getall`` method.
Returns a ``MultiDict`` container or a ``UnicodeMultiDict`` when
``charset`` is set.
"""
params = self._GET()
if self.charset:
params = UnicodeMultiDict(params, encoding=self.charset,
errors=self.errors,
decode_keys=self.decode_param_names)
return params
GET = property(GET, doc=GET.__doc__)
def _POST(self):
return parse_formvars(self.environ, include_get_vars=False)
def POST(self):
"""Dictionary-like object representing the POST body.
Most values are encoded strings, or unicode strings when
``charset`` is set. There may also be FieldStorage objects
representing file uploads. If this is not a POST request, or the
body is not encoded fields (e.g., an XMLRPC request) then this
will be empty.
This will consume wsgi.input when first accessed if applicable,
but the raw version will be put in
environ['paste.parsed_formvars'].
Returns a ``MultiDict`` container or a ``UnicodeMultiDict`` when
``charset`` is set.
"""
params = self._POST()
if self.charset:
params = UnicodeMultiDict(params, encoding=self.charset,
errors=self.errors,
decode_keys=self.decode_param_names)
return params
POST = property(POST, doc=POST.__doc__)
def params(self):
"""Dictionary-like object of keys from POST, GET, URL dicts
Return a key value from the parameters, they are checked in the
following order: POST, GET, URL
Additional methods supported:
``getlist(key)``
Returns a list of all the values by that key, collected from
POST, GET, URL dicts
Returns a ``MultiDict`` container or a ``UnicodeMultiDict`` when
``charset`` is set.
"""
params = MultiDict()
params.update(self._POST())
params.update(self._GET())
if self.charset:
params = UnicodeMultiDict(params, encoding=self.charset,
errors=self.errors,
decode_keys=self.decode_param_names)
return params
params = property(params, doc=params.__doc__)
def cookies(self):
"""Dictionary of cookies keyed by cookie name.
Just a plain dictionary, may be empty but not None.
"""
return get_cookie_dict(self.environ)
cookies = property(cookies, doc=cookies.__doc__)
def determine_browser_charset(self):
"""
Determine the encoding as specified by the browser via the
Content-Type's charset parameter, if one is set
"""
charset_match = _CHARSET_RE.search(self.headers.get('Content-Type', ''))
if charset_match:
return charset_match.group(1)
def match_accept(self, mimetypes):
"""Return a list of specified mime-types that the browser's HTTP Accept
header allows in the order provided."""
return desired_matches(mimetypes,
self.environ.get('HTTP_ACCEPT', '*/*'))
def __repr__(self):
"""Show important attributes of the WSGIRequest"""
pf = pformat
msg = '<%s.%s object at 0x%x method=%s,' % \
(self.__class__.__module__, self.__class__.__name__,
id(self), pf(self.method))
msg += '\nscheme=%s, host=%s, script_name=%s, path_info=%s,' % \
(pf(self.scheme), pf(self.host), pf(self.script_name),
pf(self.path_info))
msg += '\nlanguages=%s,' % pf(self.languages)
if self.charset:
msg += ' charset=%s, errors=%s,' % (pf(self.charset),
pf(self.errors))
msg += '\nGET=%s,' % pf(self.GET)
msg += '\nPOST=%s,' % pf(self.POST)
msg += '\ncookies=%s>' % pf(self.cookies)
return msg
class WSGIResponse(object):
"""A basic HTTP response with content, headers, and out-bound cookies
The class variable ``defaults`` specifies default values for
``content_type``, ``charset`` and ``errors``. These can be overridden
for the current request via the registry.
"""
defaults = StackedObjectProxy(
default=dict(content_type='text/html', charset='utf-8',
errors='strict', headers={'Cache-Control':'no-cache'})
)
def __init__(self, content='', mimetype=None, code=200):
self._iter = None
self._is_str_iter = True
self.content = content
self.headers = HeaderDict()
self.cookies = SimpleCookie()
self.status_code = code
defaults = self.defaults._current_obj()
if not mimetype:
mimetype = defaults.get('content_type', 'text/html')
charset = defaults.get('charset')
if charset:
mimetype = '%s; charset=%s' % (mimetype, charset)
self.headers.update(defaults.get('headers', {}))
self.headers['Content-Type'] = mimetype
self.errors = defaults.get('errors', 'strict')
def __str__(self):
"""Returns a rendition of the full HTTP message, including headers.
When the content is an iterator, the actual content is replaced with the
output of str(iterator) (to avoid exhausting the iterator).
"""
if self._is_str_iter:
content = ''.join(self.get_content())
else:
content = str(self.content)
return '\n'.join(['%s: %s' % (key, value)
for key, value in self.headers.headeritems()]) \
+ '\n\n' + content
def __call__(self, environ, start_response):
"""Convenience call to return output and set status information
Conforms to the WSGI interface for calling purposes only.
Example usage:
.. code-block:: python
def wsgi_app(environ, start_response):
response = WSGIResponse()
response.write("Hello world")
response.headers['Content-Type'] = 'latin1'
return response(environ, start_response)
"""
status_text = STATUS_CODE_TEXT[self.status_code]
status = '%s %s' % (self.status_code, status_text)
response_headers = self.headers.headeritems()
for c in self.cookies.values():
response_headers.append(('Set-Cookie', c.output(header='')))
start_response(status, response_headers)
is_file = isinstance(self.content, file)
if 'wsgi.file_wrapper' in environ and is_file:
return environ['wsgi.file_wrapper'](self.content)
elif is_file:
return iter(lambda: self.content.read(), '')
return self.get_content()
def determine_charset(self):
"""
Determine the encoding as specified by the Content-Type's charset
parameter, if one is set
"""
charset_match = _CHARSET_RE.search(self.headers.get('Content-Type', ''))
if charset_match:
return charset_match.group(1)
def has_header(self, header):
"""
Case-insensitive check for a header
"""
warnings.warn('WSGIResponse.has_header is deprecated, use '
'WSGIResponse.headers.has_key instead', DeprecationWarning,
2)
return self.headers.has_key(header)
def set_cookie(self, key, value='', max_age=None, expires=None, path='/',
domain=None, secure=None, httponly=None):
"""
Define a cookie to be sent via the outgoing HTTP headers
"""
self.cookies[key] = value
for var_name, var_value in [
('max_age', max_age), ('path', path), ('domain', domain),
('secure', secure), ('expires', expires), ('httponly', httponly)]:
if var_value is not None and var_value is not False:
self.cookies[key][var_name.replace('_', '-')] = var_value
def delete_cookie(self, key, path='/', domain=None):
"""
Notify the browser the specified cookie has expired and should be
deleted (via the outgoing HTTP headers)
"""
self.cookies[key] = ''
if path is not None:
self.cookies[key]['path'] = path
if domain is not None:
self.cookies[key]['domain'] = domain
self.cookies[key]['expires'] = 0
self.cookies[key]['max-age'] = 0
def _set_content(self, content):
if hasattr(content, '__iter__'):
self._iter = content
if isinstance(content, list):
self._is_str_iter = True
else:
self._is_str_iter = False
else:
self._iter = [content]
self._is_str_iter = True
content = property(lambda self: self._iter, _set_content,
doc='Get/set the specified content, where content can '
'be: a string, a list of strings, a generator function '
'that yields strings, or an iterable object that '
'produces strings.')
def get_content(self):
"""
Returns the content as an iterable of strings, encoding each element of
the iterator from a Unicode object if necessary.
"""
charset = self.determine_charset()
if charset:
return encode_unicode_app_iter(self.content, charset, self.errors)
else:
return self.content
def wsgi_response(self):
"""
Return this WSGIResponse as a tuple of WSGI formatted data, including:
(status, headers, iterable)
"""
status_text = STATUS_CODE_TEXT[self.status_code]
status = '%s %s' % (self.status_code, status_text)
response_headers = self.headers.headeritems()
for c in self.cookies.values():
response_headers.append(('Set-Cookie', c.output(header='')))
return status, response_headers, self.get_content()
# The remaining methods partially implement the file-like object interface.
# See http://docs.python.org/lib/bltin-file-objects.html
def write(self, content):
if not self._is_str_iter:
raise IOError, "This %s instance's content is not writable: (content " \
'is an iterator)' % self.__class__.__name__
self.content.append(content)
def flush(self):
pass
def tell(self):
if not self._is_str_iter:
raise IOError, 'This %s instance cannot tell its position: (content ' \
'is an iterator)' % self.__class__.__name__
return sum([len(chunk) for chunk in self._iter])
########################################
## Content-type and charset
def charset__get(self):
"""
Get/set the charset (in the Content-Type)
"""
header = self.headers.get('content-type')
if not header:
return None
match = _CHARSET_RE.search(header)
if match:
return match.group(1)
return None
def charset__set(self, charset):
if charset is None:
del self.charset
return
try:
header = self.headers.pop('content-type')
except KeyError:
raise AttributeError(
"You cannot set the charset when no content-type is defined")
match = _CHARSET_RE.search(header)
if match:
header = header[:match.start()] + header[match.end():]
header += '; charset=%s' % charset
self.headers['content-type'] = header
def charset__del(self):
try:
header = self.headers.pop('content-type')
except KeyError:
# Don't need to remove anything
return
match = _CHARSET_RE.search(header)
if match:
header = header[:match.start()] + header[match.end():]
self.headers['content-type'] = header
charset = property(charset__get, charset__set, charset__del, doc=charset__get.__doc__)
def content_type__get(self):
"""
Get/set the Content-Type header (or None), *without* the
charset or any parameters.
If you include parameters (or ``;`` at all) when setting the
content_type, any existing parameters will be deleted;
otherwise they will be preserved.
"""
header = self.headers.get('content-type')
if not header:
return None
return header.split(';', 1)[0]
def content_type__set(self, value):
if ';' not in value:
header = self.headers.get('content-type', '')
if ';' in header:
params = header.split(';', 1)[1]
value += ';' + params
self.headers['content-type'] = value
def content_type__del(self):
try:
del self.headers['content-type']
except KeyError:
pass
content_type = property(content_type__get, content_type__set,
content_type__del, doc=content_type__get.__doc__)
## @@ I'd love to remove this, but paste.httpexceptions.get_exception
## doesn't seem to work...
# See http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html
STATUS_CODE_TEXT = {
100: 'CONTINUE',
101: 'SWITCHING PROTOCOLS',
200: 'OK',
201: 'CREATED',
202: 'ACCEPTED',
203: 'NON-AUTHORITATIVE INFORMATION',
204: 'NO CONTENT',
205: 'RESET CONTENT',
206: 'PARTIAL CONTENT',
226: 'IM USED',
300: 'MULTIPLE CHOICES',
301: 'MOVED PERMANENTLY',
302: 'FOUND',
303: 'SEE OTHER',
304: 'NOT MODIFIED',
305: 'USE PROXY',
306: 'RESERVED',
307: 'TEMPORARY REDIRECT',
400: 'BAD REQUEST',
401: 'UNAUTHORIZED',
402: 'PAYMENT REQUIRED',
403: 'FORBIDDEN',
404: 'NOT FOUND',
405: 'METHOD NOT ALLOWED',
406: 'NOT ACCEPTABLE',
407: 'PROXY AUTHENTICATION REQUIRED',
408: 'REQUEST TIMEOUT',
409: 'CONFLICT',
410: 'GONE',
411: 'LENGTH REQUIRED',
412: 'PRECONDITION FAILED',
413: 'REQUEST ENTITY TOO LARGE',
414: 'REQUEST-URI TOO LONG',
415: 'UNSUPPORTED MEDIA TYPE',
416: 'REQUESTED RANGE NOT SATISFIABLE',
417: 'EXPECTATION FAILED',
500: 'INTERNAL SERVER ERROR',
501: 'NOT IMPLEMENTED',
502: 'BAD GATEWAY',
503: 'SERVICE UNAVAILABLE',
504: 'GATEWAY TIMEOUT',
505: 'HTTP VERSION NOT SUPPORTED',
}

View File

@@ -0,0 +1,38 @@
Metadata-Version: 1.0
Name: PasteDeploy
Version: 1.5.0
Summary: Load, configure, and compose WSGI applications and servers
Home-page: http://pythonpaste.org/deploy/
Author: Alex Gronholm
Author-email: alex.gronholm@nextday.fi
License: MIT
Description: This tool provides code to load WSGI applications and servers from
URIs; these URIs can refer to Python Eggs for INI-style configuration
files. `Paste Script <http://pythonpaste.org/script>`_ provides
commands to serve applications based on this configuration file.
The latest version is available in a `Mercurial repository
<http://bitbucket.org/ianb/pastedeploy>`_ (or a `tarball
<http://bitbucket.org/ianb/pastedeploy/get/tip.gz#egg=PasteDeploy-dev>`_).
For the latest changes see the `news file
<http://pythonpaste.org/deploy/news.html>`_.
Keywords: web wsgi application server
Platform: UNKNOWN
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 2.5
Classifier: Programming Language :: Python :: 2.6
Classifier: Programming Language :: Python :: 2.7
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.1
Classifier: Programming Language :: Python :: 3.2
Classifier: Topic :: Internet :: WWW/HTTP
Classifier: Topic :: Internet :: WWW/HTTP :: Dynamic Content
Classifier: Topic :: Internet :: WWW/HTTP :: WSGI
Classifier: Topic :: Internet :: WWW/HTTP :: WSGI :: Middleware
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Framework :: Paste

Some files were not shown because too many files have changed in this diff Show More