# -*- coding: utf-8 -*- """Functions used to create function and method signatures Copyright © 2005-2012, Anders Andersen, University of Tromsø, Norway. See http://www.cs.uit.no/~aa/dist/tools/noop/COPYING (../COPYING) for details. Function and method signatures ============================== The following functions are used to add signatures and type checking to functions and methods. The module implements the decorator `signature` that adds a signature (and type checking) to functions. Some limited type cheking is done at define time (number of arguments and the type of default values). At call time the arguments, the return value and any exceptions of a function/method call can be checked (call time type checking is performed if the `__callTimeTC__` attribute of function/method is `True`). Type specification ------------------ The `signature` decorator is used to specify the signature of a function/method. A signature includes type information about the arguments and the return values, and a list of possible exceptions raised by the function. The actual type specification needs a language to express such specifications. Our approach is an extension of the type language provided by the standard Python module `types`. In this language an integer has the type `int`: >>> type(1) is int For a complete list of names for types see Built-in Types and the `types` module in the Python Library Reference. The type of an object is its class. The rules for sub- and superclasses follows the same rules as the built in Python function `isinstance`. The classes and standard type names are not enough for the signature type specification language. The types `list`, `tuple` and `dict` does not allow you to specify more details about the elements (and keys) in these composite types. We have extended the type sepcification language with composite type specification. The type of a tuple can include the type of each element. The tuple `(1, 'foo', 2.3)` has the non-composite type `tuple` and the composite type `(int, str, float)`. The type of a list can include the type of all elements. The list `[1, 4, 7, 8]` has the non-composite type `list` and the composite type `[int]`. The type of a dictionary can either be just `dict`, or it can include the type of the key and the value, or it can contain a set of specific keys and the type of their values. The dictionary `{'ID': 212, 'GID': 100}` has the composite type `{str: int}`. The dictionary `{'id': 42, 'sh': 'bash'}` has the composite type `{'id': int, 'sh': str}`. The elements of a composite type can be a composite type. For example, the third element of a three-tuple can be a list of integers: `(int, str, [int])`. Finally we add a set of new type constructs to make it possible to specify more flexible type specifications in the language. `opt(t)` matches a value of type `t` or no value (we use the term `missing`). `one(t1, t2, ..., tn)` matches a value either of type `t1`, or of type `t2`, ..., or of type `tn`. `pred(t, p)` matches a value of type `t` if the predicate `p` is true too. The predicate is a function that take one argument (the value) and returns `True` or `False` based on this value. We also provide the type construct `missing` that means no argument, but I think this is only useful in complex type specifications that are using the `isarg` construct (see below). `xargs` matches an argument tuple (eg. *args), and `xxkw` matches an agrument dictionary (eg. **kw). The Python type `object` can be used to match a single value of any types (everything are objects in Python). The last two new constructs refere to other arguments (by number or name). `arg(1)` referes to the first argument, and the meaning is the same type as the first argument. For named arguments `arg('id')` referes to the type of the argument named `id`. `isarg` is a mapping type construct. `isarg(1, {int: missing, [int]: int})` says that if the first argument is an integer, then this type is `missing` (meaning no arguments), and if the first argument is a list of integers, the this type is an integer. `isarg` can also refere to the argument by name of we have named arguments. Usage ----- Some examples of the usage of the signature decorator: >>> @signature((str, opt(int)), [str], []) >>> def listit(s, n=1): >>> return [s] * n """ # CVS/RCS information _file = r"$RCSfile: signature.py,v $" _revision = r"$Revision: 2.40 $" _date = r"$Date: 2012-10-17 13:13:05 $" _state = r"$State: Exp $" _author = r"$Author: aa $" _log = r""" $Log: signature.py,v $ Revision 2.40 2012-10-17 13:13:05 aa Added comments and argument descriptions. Revision 2.39 2012-10-09 09:15:22 aa Also save module in the replacement function. Revision 2.38 2012-10-09 09:07:14 aa Added `namecallable()` function. Revision 2.37 2012-10-08 19:52:28 aa Added copyright notice. Revision 2.36 2012-09-28 09:55:35 aa Corrected bug in `_tname()`: 'dict_items' object does not support indexing. Revision 2.35 2012-09-28 09:23:00 aa Updated `_tname()` to handle more composite OOPP types. Revision 2.34 2012-09-26 23:04:18 aa Added the `_tname()` function to correctly create the name of a type. Revision 2.33 2012-08-30 22:20:12 aa Use name attribute (`__name__`) of types when reporting type mismatch. Revision 2.32 2012-08-30 10:40:33 aa Added the name attribute to `arg` and `isarg` and shorten the argument specific parts of the names. Revision 2.31 2012-08-29 19:05:39 aa Added a name attribute to type specifications. Revision 2.30 2012-08-29 11:10:56 aa Replaced 'str()' with 'repr()' when producing type error messages. Revision 2.29 2010-10-05 14:48:15 aa Removed the `exceptions` decorator and fixed the generator case. Revision 2.28 2010/09/27 13:18:42 aa Better handling of functions vs methods. Revision 2.27 2010/09/08 16:30:55 aa Gone back to test the name of the first argument ("self") to recognize bound methods since an instance of the class is not yet created when this test is performed. Revision 2.26 2010/09/08 15:34:54 aa Added Python 3 check. Revision 2.25 2010/09/06 13:28:17 aa Second try to correct the implementation of case 1 in `checkargs`. Revision 2.24 2010/09/06 13:09:20 aa Corrected `checkargs` when optional agruments were not provided (case 1) and improved method/function test in `funcinfo` (the first argument of a method do not have to be named `self`). Revision 2.23 2010/09/03 11:05:37 aa Renamed `istype` to `check`. Revision 2.22 2010/09/01 22:38:42 aa Removed the previous correction in the `_addsig` function. Not needed in Python 3. Revision 2.21 2010/09/01 22:34:47 aa Corrected the test for annotations in the `_addsig` function. Revision 2.20 2010/09/01 22:00:27 aa Corrected an error in the `checkargs` function: the no more arguments test was wrong. Revision 2.19 2010/09/01 13:32:10 aa Corrected argument handling in the `istype` method. Revision 2.18 2010/09/01 13:26:45 aa Added missing `self` argument to new `istype` method in `typespec`. Revision 2.17 2010/09/01 13:25:51 aa Added the `istype` method to the `typespec` class. Revision 2.16 2010/09/01 12:35:14 aa Reintroduced the any type specification using the name `whatever`. Revision 2.15 2010/08/20 12:24:57 aa Added support for signature decorators without arguments. Revision 2.14 2010/03/22 09:22:47 aa Removed the type `any`. You should use the Python type `object` instead. Revision 2.13 2010/03/18 17:48:31 aa Fixed issues with missing arguments and string representation of some type specifications. Revision 2.12 2010/03/18 16:04:16 aa Added support for the Sig class. Revision 2.11 2010/02/09 12:31:01 aa Added the `Sig` class. Revision 2.10 2010/01/22 17:27:58 aa Added default argument `missing` to `opt` call function. Revision 2.9 2010/01/07 10:58:59 aa Added new type of signature/exceptions specifications. Revision 2.8 2009/12/04 17:17:54 aa Fiexed a few issues where None are treated different from empty list and a few other minor errors. Revision 2.7 2009/12/04 04:54:57 aa Fixed annotations (had to interpret annotations earlier). Revision 2.6 2009/12/03 23:14:31 aa Restructured the decorator code (signature and exceptions) and added annotation support. Revision 2.5 2009/12/03 16:05:32 aa Changed rest of _typecheck to an argument and not a return value. Revision 2.4 2009/12/02 22:38:49 aa Corrected a typo. Revision 2.3 2009/12/02 17:18:48 aa Corrected yet another Python 3 error: dict has_key() no longer exits. Revision 2.2 2009/12/02 17:13:42 aa Corrected another Python 3 error: dict keys() no longer returns a list. Revision 2.1 2009/12/02 16:57:39 aa Corrected type names to Python 3 names. Revision 2.0 2009/12/02 15:41:18 aa Moved to Python 3 syntax. Revision 1.36 2009/08/26 09:34:09 aa Added the exceptions decorator. Revision 1.35 2005/12/06 23:27:19 aa Added more comments. Revision 1.34 2005/12/04 12:13:41 aa Added some comments (in `_initArg`) and corrected a return bug/typo. Revision 1.33 2005/12/03 23:45:50 aa Corrected a few bugs in the `arg`/`isarg` implementation in `checkfunc`. Revision 1.32 2005/12/03 23:26:59 aa Made more functions private to the module (`initArg` and `stripX...`), moved code from `signature` to create the `funcinfo` function, renamed `typecheck` to `_typecheck` and created a new `typecheck` that also calls the delayed check (now made private and called `_delayedcheck`), and finally added support for `arg`/`isarg` in `checkfunc`. More debugging might be needed. Revision 1.31 2005/12/02 14:55:35 aa Added different exceptions for argument, return value, and exception errors. Revision 1.30 2005/12/02 11:47:23 aa Added catching of exceptions (only allow listed exceptions). Revision 1.29 2005/12/01 17:57:06 aa Minor typo corrected. Revision 1.28 2005/12/01 17:55:36 aa This is a large check-in mostly concerned getting `arg` and `isarg` to work correctly. This involves a lot of changes in existing functions and many new functions (and classes). Also some documentation added. Revision 1.27 2005/11/30 17:05:58 aa Added `arg` and `isarg` and comments/documentation. Revision 1.26 2005/11/30 11:02:38 aa Updated documentation og comments. Revision 1.25 2005/11/01 19:51:01 aa RST stuff removed. Revision 1.24 2005/11/01 19:47:53 aa Added a lot of rst stuff, but it will be removed in the next version. Revision 1.23 2005/10/25 18:16:29 aa Corrected missing exception catch in checkfunc (xname and xxname). Revision 1.22 2005/10/25 17:16:14 aa Cleaned up checkfunc implementation (including adding dictionary signature support). Revision 1.21 2005/10/25 05:44:18 aa The checkfunc function implemented but not debugged or extensively tested. Revision 1.20 2005/10/25 04:07:18 aa Fixed stripXsig and stripXargs, and added initial checkfunc. Revision 1.19 2005/10/24 20:12:29 aa Added xargs and xxkw support to signatures. Revision 1.18 2005/10/24 16:38:24 aa The unpacking/packing of args and kw in checkargs is removed. Revision 1.17 2005/10/23 06:18:12 aa Made it possible to select/change call time type checking at runtime (addsig). Revision 1.16 2005/10/22 20:30:34 aa Corrected a few errors in typecheck (the dictionary part). Revision 1.15 2005/10/22 19:12:34 aa Added list and dictionary type specification checks in typecheck. Revision 1.14 2005/10/21 22:29:01 aa Implemented case 2 (dictionary) in checkargs and dig out more information about the function in addsig in signature. Revision 1.13 2005/10/20 22:22:50 aa Corrected typo. Revision 1.12 2005/10/20 19:48:28 aa Added comments. Revision 1.11 2004/12/03 14:26:45 aa Added named arguments test in checkargs. Revision 1.10 2004/10/24 12:17:29 aa Final fixes for type cheking. Composite types to come. Revision 1.9 2004/10/23 05:24:58 aa Corrected some type checking errors. Revision 1.8 2004/10/22 19:29:50 aa Implemented type checking for simple data types (and tuples). Revision 1.7 2004/10/22 04:10:40 aa Added the implementation of type checking for any, opt and one. Revision 1.6 2004/10/21 22:49:11 aa Added __call__, repr and str to the any, opt and one class + comments. Revision 1.5 2004/10/21 16:17:53 aa Added comments and the typespec class, and renamed checksig checkargs. Revision 1.4 2004/10/21 05:59:38 aa Also save other attributes of a fuction (document string and so on). Revision 1.3 2004/10/21 05:15:58 aa Started implementing the signature decorator. Revision 1.2 2004/10/20 22:05:19 aa Minor corrections (added missing import statements and similar). Revision 1.1 2004/10/20 21:56:52 aa Created the signature module with an signature function doing nothing. """ # Load some standard python libraries from types import * import sys # Python 3 only! assert sys.version_info[0] > 2, \ "The NOOP module \"%s\" is Python 3 (or greater) only!" % (__name__,) # Some module specific values (might be modified by the user of the module) callTimeTC = True # call-time type checking # An exception raised if function or call does not match signature class SignatureError(TypeError): pass # An exception raised if call argument does not match signature class ArgumentError(SignatureError): pass # An exception raised if call return value does not match signature class ReturnValueError(SignatureError): pass # An exception raised if call return value does not match signature class ExceptionError(SignatureError): pass # An exception raised when a type checking is not performed class NotChecked(SignatureError): pass def _tname(t): """Return name of type Returns the name of a type. When the type is a list, tuple or dictionary, the type information is recursivly parsed. t -- the type to name """ if hasattr(t, '__name__'): return t.__name__ elif type(t) is tuple: return '(' + ''.join(map(lambda x: _tname(x) + ",", t))[:-1] + ')' elif type(t) is list: return '[' + ''.join(map(lambda x: _tname(x) + ",", t))[:-1] + ']' elif type(t) is dict: if len(t) == 1: k,v = list(t.items())[0] return '{%s:%s}' % (_tname(k), _tname(v)) else: return '{' + ''.join( map(lambda x: repr(x[0]) + ':' + _tname(x[1]) + ',', t.items()))[:-1] + '}' else: return str(t) class typespec: """Type specification The base class for all noop specific type specification marks. """ def __init__(self): """Initialise the type specification Initialise the type specification instance with a name. """ self.__name__ = self.__class__.__name__ def __call__(self, value=None): """Type checking This one allways fails since it is not a valid type specification. You should use a class that has `typespec` as its base class and is redefining this method. """ raise SignatureError("%s is not a valid type specification" % (self,)) def __repr__(self): """The repr string The repr string of typespec. """ return "" def check(self, *args): """Is the value of this type Returns `True` if `value` matches the type, otherwise `False`. args -- the value to check against the type """ try: self(*args); return True except SignatureError: return False # The str string is equal to the repr string __str__ = __repr__ # This means missing or no argument (it is a hack, but a good one) class missing(typespec): def __repr__(self): return "" __str__ = __repr__ missing = missing() # These are for * (xargs) and ** (xxkw) arguments class xargs(typespec): def __repr__(self): return "" __str__ = __repr__ xargs = xargs() class xxkw(typespec): def __repr__(self): return "" __str__ = __repr__ xxkw = xxkw() class whatever(typespec): """Any argument allowed Used to specify that any argument or return value is valid. This type specifier can also be used inside a more complex type specification (`[whatever]` is a list with elements of any type). """ def __call__(self, value=missing, rest=[]): """Type checking The actual `whatever` type checking. This always pass if an argument of any type is given. value -- the value to check when type checking rest -- used to keep non-checked arguments (low level stuff) """ if (value == missing): raise SignatureError("whatever is one argument of any type") def __repr__(self): """The repr string The repr string of the type whatever. """ return "" # The str string is equal to the repr string __str__ = __repr__ # Hmm, I think I prefere "whatever" instead of "whatever()" in type specs whatever = whatever() class pred(typespec): """Add a predicate to a type First check that the argument is of the specified type, then check if it fullfills the predicate. """ def __init__(self, orgType, pred): """Initialise the predicate Save the type and its predicate. orgType -- The type to match when type checking pred -- The predicate to match when type checking """ self.type = orgType self.pred = pred self.__name__ = self.__class__.__name__ + "(%s,%s)" % ( _tname(orgType), _tname(pred)) def __call__(self, value=missing, rest=[]): """Type checking Perform the type checking. First check the type of the value, then the predicate. If the predicate is false raise an error. value -- the value to check when type checking rest -- used to keep non-checked arguments (low level stuff) """ _typecheck(self.type, value, rest) if not self.pred(value): raise SignatureError("predicate not fulfilled: %s" % (self.__name__,)) def _repr_or_str_(self, m): """Help method for repr and str A help method used to implement repr and str for the object. m -- either `repr` or `str` """ return "" # repr and str are implemented by _repr_or_str_ __repr__ = lambda self: pred._repr_or_str_(self, repr) __str__ = lambda self: pred._repr_or_str_(self, str) class opt(typespec): """The argument is optional The given argument is optional, but if its is given it is of the type specified. In Python this means that the argument has a default value (or the value None). """ def __init__(self, optType): """Initialise the opt type Save the optional type. optType -- The optional type to match when type checking """ self.type = optType self.__name__ = self.__class__.__name__ + "(%s)" % (_tname(optType),) def __call__(self, value=missing, rest=[]): """Type checking The actual `opt` type checking. It pass if `value` is not missing or if the specified type matches the argument. value -- the value to check when type checking rest -- used to keep non-checked arguments (low level stuff) """ if value != missing: _typecheck(self.type, value, rest) def _repr_or_str_(self, m): """Help method for repr and str A help method used to implement repr and str for the object. m -- either `repr` or `str` """ return "" # repr and str are implemented by _repr_or_str_ __repr__ = lambda self: opt._repr_or_str_(self, repr) __str__ = lambda self: opt._repr_or_str_(self, str) class one(typespec): """The argument is of one of the listed types The argument or return value is of one of the listed types. """ def __init__(self, *optType): """Initialise the one type Save the tuple of legal types. optType -- the possible types to match when type checking """ if len(optType) < 1: raise SignatureError("one constructor needs at least one argument") self.types = optType tnames = "".join(map(lambda x: _tname(x) + ",", optType))[:-1] self.__name__ = self.__class__.__name__ + "(%s)" % (tnames,) def __call__(self, value=missing, rest=[]): """Type checking The actual `one` type checking. It pass if at least one of the arguments matches the given type. value -- the value to check when type checking rest -- used to keep non-checked arguments (low level stuff) """ for thetype in self.types: try: _typecheck(thetype, value, rest) return except SignatureError: continue raise SignatureError("%s does not match %s" % (repr(value), self.__name__)) def _repr_or_str_(self, m): """Help method for repr and str A help method used to implement repr and str for the object. m -- either `repr` or `str` """ s = "" # repr and str are implemented by _repr_or_str_ __repr__ = lambda self: one._repr_or_str_(self, repr) __str__ = lambda self: one._repr_or_str_(self, str) def namecallable(func): """The name of the type of the callable object The function returns the name of type the callable object `func`. For a bound method the name is 'method', for a class method the name is 'classmethod', for an unbound function the name is 'function', and for all other callable objects the name is just 'type'. Raises a `TypeError` exception if `func` is not callable. func -- the callable object where we want to name the type """ # Is it callable: if callable(func): # It has to have a class if hasattr(func, "__class__"): # Either 'method', 'classmethod', 'function', 'generator', # 'builtin_function_or_method', or 'type' if hasattr(func.__class__, "__name__"): return func.__class__.__name__ # No class or name found, guess it is 'type' return 'type' # Not callable else: raise TypeError("%s not callable." % (str(func),)) def ismethod(func, san="self"): """Is it a method? Is `func` a method? This will work both if you use the class or object method (for a non-static method `m` defined in class `C` this function will return `True` both for `C.f` and `C().f`). The optional argument `san` is the self argument name (the reference to the object instance). It is common practice in Python to name this 'self' in methods. In a class method the name 'cls' is often used, but we do not need to test this since a class method always is of type `MethodType`, even before an instance is created. For static methods this function will always return `False` (but static method should not have a first argument with a name matching the `san` argument). func -- is this a method san -- the name used for the `self` arguments of object methods """ return ( ( hasattr(func, "__funcdetails__") and # Allready tested this? func.__funcdetails__["bound"]) # See `funcinfo` below or ( type(func) is MethodType) # Non-stat or cls method or ( func.__code__.co_argcount > 0 and # At least one argument func.__code__.co_varnames[0] == san) # "self" first argument ) def _findargtype(thetype, thevalue): """Find the actual type Find the actual type of an argument based on its type specification and the value of the arguement. Both `opt` and `one` specifications can be nested. thetype -- the type, can be any Python and NOOP type thevalue -- the value that should be matched to an explicit type """ # If `thetype` is a Python type if type(thetype) is type: return thetype # The type is whatever type elif isinstance(thetype, whatever.__class__): # Use the actual type of the value return type(thevalue) # The type is one of a list (`one`) elif isinstance(thetype, one): # Find which one (and ignore it if no one matches) for t in thetype.types: rest = [] try: _typecheck(t, thevalue, rest) except SignatureError: continue else: return _findargtype(t, thevalue) else: return None # The type is optional (`opt`) elif isinstance(thetype, opt): # No value if thevalue == missing: return missing # A value else: return _findargtype(thetype.type, thevalue) # The type is indirect since it is refered to by another argument (`_ref`) elif isinstance(thetype, _ref): return _findargtype(thetype.reftype, thevalue) class arg(typespec): """The argument type is taken from another argument The argument or return value type is eqal to another argument refered to be a number or a name. """ def __init__(self, argref): """Save argument reference Save the argument number or name. argref -- the name or number of the argument referred to """ self.argref = argref self.__name__ = self.__class__.__name__ + "(%s)" % (str(argref),) def init(self, reftype, value): """A second level initialization Get the type of the refered argument. This is called during type check of the refered arguments. reftype -- the NOOP type value -- the value to find matching type for """ self.thetype = _findargtype(reftype, value) def reset(self): """Reset specification Remove the type found at second level initialization. """ del self.thetype def __call__(self, value, rest=[]): """Type checking The actual `arg` type checking. It passes if the argument type is the same as the type of the refered argument. value -- the value to check when type checking rest -- used to keep non-checked arguments (low level stuff) """ if hasattr(self, "thetype"): _typecheck(self.thetype, value, rest) else: raise NotChecked("Type constructor that needs second level init") def __repr__(self): """The repr string The repr string of the type `arg`. """ return "" # The str string is equal to the repr string __str__ = __repr__ class isarg(arg): """The argument type is mapped from another argument The argument or return value type is mapped from the type of another argument refered to be a number or a name. """ def __init__(self, argref, tmap): """Save argument reference and map Save the argument number or name, and the mapping from this argument to the given type. argref -- the argument number or name tmap -- the mapping from one type to another type """ self.argref = argref self.map = tmap tmn = "".join(map( lambda x: _tname(x[0]) + ":" + _tname(x[1]) + ",", tmap.items()))[:-1] self.__name__ = self.__class__.__name__ + "(%s,{%s})" % ( str(argref), tmn) def init(self, reftype, value): """A second level initialization Get the type of the refered argument. reftype -- the NOOP type value -- the value to find matching type for """ arg.init(self, reftype, value) try: self.thetype = self.map[self.thetype] except KeyError: raise SignatureError("Refered argument type %s" % ( repr(self.thetype),) + " not found in map (isarg)") def _repr_or_str_(self, m): """Help method for repr and str A help method used to implement repr and str for the object. m -- either `repr` or `str` """ return "" # repr and str are implemented by _repr_or_str_ __repr__ = lambda self: isarg._repr_or_str_(self, repr) __str__ = lambda self: isarg._repr_or_str_(self, str) class _ref(typespec): """A refered argument Any arguments that are refered to in the signature (using `arg` or `isarg`) are changed to this type. The reason is that only during call time type checking, the actual type of this argument is decided (the type is either `opt`, `one` or `whatever`). The referee has to be updated with this type information. """ def __init__(self, reftype, referee): """Initialize the refered arguments Save the actual type, and a reference to the referee (see `arg` and `argsig` above). reftype -- the actual type (`opt`, `one` or `whatever`) referee -- argument/return refering to this type """ self.reftype = reftype self.referee = referee self.__str__ = reftype.__str__ self.__repr__ = reftype.__repr__ def __call__(self, value, rest=[]): """Perform type checking Do the second level initialization of the referee and the perform the type checkin. value -- the value to check when type checking rest -- used to keep non-checked arguments (low level stuff) """ self.referee.init(self.reftype, value) _typecheck(self.reftype, value, rest) def _typecheck(thetype, thevalue, rest): """Performing type checking This is the actual function performing the generic type checking. A `SignatureError` is raised if a type error is found. thetype -- the type to match thevalue -- the value to check when type checking rest -- used to keep non-checked arguments (low level stuff) """ # A type of `None` only accepts the value `None` (important for # return values) if thetype == None: if thevalue != None: raise SignatureError("None is the only valid None type: %s" % ( thevalue,)) # Only valid value of the type `missing` is `missing` elif thetype == missing: if thevalue != missing: raise SignatureError("No argument is the only valid missing type") # Check each type in the type tuple against its matching value in thevalue elif type(thetype) is tuple: # Then the thevalue has to be a tuple type too if not type(thevalue) is tuple: raise SignatureError("%s is not valid tuple type" % (thevalue,)) # The type tuple has to be at least as long as the value tuple if len(thetype) < len(thevalue): raise SignatureError("maximum %d arguments: %s is not %s" % ( len(thetype), thevalue, "(" + "".join(map(lambda x: _tname(x) + ",", thetype))[:-1] + ")")) # Each type (element) in the type tuple i = 0 for t in thetype: # The matching value in thevalue try: a = thevalue[i] # Argument missing except IndexError: a = missing # Perform the type checking of each element _typecheck(t, a, rest) i += 1 # If the type is a typespec object, let the object do the type checking elif (isinstance(thetype, typespec) or isinstance(thetype, whatever.__class__)): try: thetype(thevalue, rest) # Delayed type checking (see `delayedcheck`) except NotChecked: rest.append((thetype, thevalue)) # If the type is a list elif type(thetype) is list: # The the value has to be a list type too if not type(thevalue) is list: raise SignatureError("%s is not valid list type" % ( repr(thevalue),)) # The value should be a list of the given type if len(thetype) == 1: for v in thevalue: _typecheck(thetype[0], v, rest) # Python accepts list with element of different types, but use list else: raise SignatureError("%s is not a valid type specification" % ( repr(thetype),)) # If the type is a dictionary elif type(thetype) is dict: # The value has to be a dictionary too if not type(thevalue) is dict: raise SignatureError("%s is not valid dict type" % ( repr(thevalue),)) # The value should be a dictionary with typed key, value pairs if len(thetype) == 1: key = list(thetype.keys())[0] val = thetype[key] for k, v in thevalue.items(): _typecheck(key, k, rest) _typecheck(val, v, rest) # The value should be a dictionary with given keys else: # The type should be at least as long as the value if len(thetype) < len(thevalue): raise SignatureError("too long dictionary: %s is not %s" % ( thevalue, "{" + "".join(map( lambda x: repr(x[0]) + ":" + _tname(x[1]) + ",", thetype.items()))[:-1] + "}")) # Check eack key-value pair nonused = set(thevalue.keys()) for key, val in thetype.items(): # The matching value in the value try: v = thevalue[key] nonused.remove(key) # Is missing (could be optional) except KeyError: v = missing # Is it a matching value _typecheck(val, v, rest) # Any unexpected elements in the value dictionary? if len(nonused) > 0: raise SignatureError("unexpected keys %s in dictionary" % ( repr(tuple(nonused)),)) # OK, this is the test that checks if the given value is of the given type elif not isinstance(thevalue, thetype): if thevalue == missing: msg = "missing argument: " + _tname(thetype) else: msg = "%s is not a %s" % (repr(thevalue), _tname(thetype)) raise SignatureError(msg) def _delayedcheck(rest): """Delayed type checking Delayed type checking is performed on type specifications refering to the type of other arguments where the actual type of this other argument was not known last time we tried to perform this type checking. rest -- list of ignored type-checked values """ # Continue until all are checked and as long as we have progress while True: # Check the rest one by one length = len(rest) for i in range(length): # If the type check is performed (not delayed), remove it newrest = [] _typecheck(*rest[i], rest=newrest) if not newrest: del rest[i] # Are we done? if len(rest) == 0: break # Do we have progress? elif not len(rest) < length: raise SignatureError("No progress on delayed typechecking") def typecheck(thetype, thevalue): """Perform type checking Check if the value matches the type. Two steps because of the `arg` and `isarg` types (we might have to delay these checks). thetype -- the type to match thevalue -- the value to check when type checking """ rest = [] _typecheck(thetype, thevalue, rest) _delayedcheck(rest) def _stripXsig(xname, xxname, argsig): """Remove * and ** arguments from signature Removes `xargs` and `xxkw` from arguments signature. xname -- the name of args argument xxname -- the name of kw arguments argsig -- the signature """ # If dictionary argument signature if type(argsig) is dict: if xname in argsig or xxname in argsig: argsig = argsig.copy() if xname in argsig: del argsig[xname] if xxname in argsig: del argsig[xxname] # If tuple argument signature else: if len(argsig) > 0 and argsig[-1] == xxkw: argsig = argsig[:-1] if len(argsig) > 0 and argsig[-1] == xargs: argsig = argsig[:-1] # Returns the stripped argument signature return argsig def _stripXargs(sf, args, kw): """Remove * and ** elements from argument Removes `xargs` and `xxkw` elements from arguments (both args and kw). sf -- function/method args -- args arguments kw -- kw arguments """ # So we don't have to recalculate it lensig = len(sf.__funcdetails__["argsig"]) lenargs = len(args) # Does the function have xargs? if sf.__funcdetails__["xname"]: if lensig < lenargs: args = args[:lensig] # Does the function have xxkw? oldkw = kw; kw = {} if sf.__funcdetails__["xxname"]: for name in sf.__funcdetails__["argnames"][lenargs:]: if name in oldkw: kw[name] = oldkw[name] # Returns stripped arguments return args, kw def checkargs(sf, args, kw): """Check the types of the arguments This function is performing type checking on the arguments to a function. It is using the `typecheck` function, but it might has to reorganise the arguments before `typecheck` is called. sf -- function/method args -- args arguments kw -- kw arguments """ # Only on signature decorated functions if not hasattr(sf, "__signature__"): raise AttributeError("%s is not decorated with a signature" % ( sf.__name__,)) # Unpack info argsig = sf.__funcdetails__["argsig"] argnames = sf.__funcdetails__["argnames"] # The self argument isn't included in type checking if sf.__funcdetails__["bound"]: if len(args) > 0: args = args[1:] elif "self" in kw: # Remove this code? (`self` isn't in `kw`) kw = kw.copy() del kw["self"] else: raise SignatureError("self argument missing") # Strip off xargs and xxkw if sf.__funcdetails__["xname"] or sf.__funcdetails__["xxname"]: args, kw = _stripXargs(sf, args, kw) # Need this lenargs = len(args) # Case 1: argsig as tuple and arguments as mixture of args and kw if type(argsig) is tuple and lenargs < len(argnames): # We create a new argument based on args and kw arglist = list(args) nonused = set(kw.keys()) # Go through the arguments that should be in kw (not in args) for name in argnames[lenargs:]: # Is the argument in kw (nonused is the set of kw names not used)? if name in nonused: # Add it (the value) to the argument list arglist.append(kw[name]) # Remove the name from the set of non-used names nonused.remove(name) # No more arguments, meaning that all names are used if len(nonused) != 0: raise SignatureError("Given argument(s) %s not in %s" % ( repr(tuple(nonused)), repr(argnames))) # Perform the type checking typecheck(argsig, tuple(arglist)) # Case 2: Named arguments, check the name and its type elif type(argsig) is dict: # To many arguments? if len(args) + len(kw) > len(argsig): raise SignatureError( "To many arguments, %d expected and %d given" % ( len(argsig), len(args) + len(kw))) # kw arguments not yet tested nonused = set(kw.keys()) # Check every type spec in dictionary for i in range(len(argsig)): # This is used a lot name = argnames[i] # First the args arguments if i < len(args): typecheck(argsig[name], args[i]) # Then the kw arguments else: # Is the argument in kw? try: typecheck(argsig[name], kw[name]) nonused.remove(name) # Is it OK that it is missing? except KeyError: typecheck(argsig[name], missing) # All kw arguments should be checked now if len(nonused) > 0: raise SignatureError("Given argument(s) %s not in %s" % ( repr(tuple(nonused)), repr(argnames))) # Case 3: Just perform the type checking (without modifying the arguments) else: # This asumes no kw arguments if len(kw) == 0: typecheck(argsig, args) else: raise SignatureError("%s + %s doesn't match %s" % ( repr(args), repr(kw), repr(argsig))) def funcinfo(f): """Retrieve information about the function Collect information about the given function/method/generator and its arguments. Method is type 0, function is type 1, and generator is type 2. f -- function, method, generator """ # Collect information in this dictionary info = {} # Normal arguments (a, b, k=1, l=2) info["numargs"] = f.__code__.co_argcount info["argnames"] = f.__code__.co_varnames[:info["numargs"]] if f.__defaults__: info["argdefaults"] = f.__defaults__ else: info["argdefaults"] = () # The "*arguments" syntax? i = info["numargs"] if f.__code__.co_flags & 0x04: # Bit 0x04 info["xname"] = f.__code__.co_varnames[i] i += 1 else: info["xname"] = "" # The "**keywords" syntax? if f.__code__.co_flags & 0x08: # Bit 0x08 info["xxname"] = f.__code__.co_varnames[i] else: info["xxname"] = "" # Generator? 32 (0x20) is generator if f.__code__.co_flags & 0x20: info["generator"] = True else: info["generator"] = False # The self argument of methods (and class argument of class # methods) is not included in type checking. This is a hack since # we conclude that functions with first argument named "self" are # bound methods. We have to do it this way since it is no other # way to see the difference between a non-static and a staic # function of a class before an instance of the class is created. # Class methods are of type `MethodType` before an instance of the # class is created, so in that case we do not have to check the # name of the first argument (it is common to name it "cls"). if ismethod(f): info["bound"] = True info["numargs"] -= 1 info["argnames"] = info["argnames"][1:] else: info["bound"] = False # Return info return info def checkfunc(f, argsig): """Check the implementation against the signature Check the function implementation against the argument signature. f -- function argsig -- agrument signature """ # None is an empty signature if argsig == None: lenargsig = 0 else: lenargsig = len(argsig) # Get info about the function info = funcinfo(f) # Save these for efficiency lenargs = len(info["argnames"]) lendefaults = len(info["argdefaults"]) # The signature has to have all arguments (and might include xargs/xxkw) if lenargsig < lenargs: raise SignatureError("Signature does not match impl: %s <-> %s" % ( repr(argsig), repr(info["argnames"]))) # Check each argument for i in range(lenargs): # If signature is a dictionary, check name and fetch type if type(argsig) is dict: try: sig = argsig[info["argnames"][i]] except KeyError: raise SignatureError("%s not in %s" % ( info["argnames"][i], repr(argsig))) # Signature is a tuple, fetch type else: sig = argsig[i] # Arguments without default value if i < lenargs - lendefaults: # Arguments without default value can not be optional if isinstance(sig, opt): raise SignatureError( "No default value on optional argument: %s is not %s" % ( info["argnames"][i], repr(sig))) # Arguments with a default value else: # Get the default value and check its type default = info["argdefaults"][i-(lenargs-lendefaults)] if default != None: # This is a litle bit tricky, but the general case are # much easier (see below) if isinstance(sig, arg): # Impossible, but for robustness check it if not hasattr(sig, "argref"): raise SignatureError( "Reference missing (arg/isarg)") # Named argument reference needs dictionary signature if type(argsig) is dict and \ type(sig.argref) is str: # Get type of refered arguments (if it exists) try: reftype = argsig[sig.argref] except KeyError: raise SignatureError( "Reference %s " % (repr(sig.argref),) + \ "(arg/isarg) not found in signature") # Find its default value (by position of name) for j in range(lenargs): # Found it (if it is not there we can not get # its value and we can not check that the # default value of the `arg`/`isarg` matches # it) if info["argnames"][j] == sig.argref: try: refvalue = info["argdefaults"][ j-(lenargs-lendefaults)] except IndexError: refvalue = missing break # Not found (break never reached) else: raise SignatureError( "Reference %s (arg/isarg) not found" % ( repr(sig.argref),)) # Number argument reference needs tuple signature elif type(argsig) is tuple and \ type(sig.argref) is int: # Get the refered argument type (if it exists) try: reftype = argsig[sig.argref - 1] except IndexError: raise SignatureError( "Reference %s (arg/isarg) not found" % ( repr(sig.argref),)) # Get its value (if it exists) try: refvalue = info["argdefaults"][ sig.argref -1 - (lenargs-lendefaults)] except IndexError: refvalue = missing # Missmatch between reference and signature type else: raise SignatureError( "Reference %s (arg/isarg) nor found" % ( repr(sig.argref),)) # Use the refered type (maybe I should do this in # all `arg`/`sigarg` cases?) if refvalue == missing: typecheck(reftype, default) # Get the specific type (since we have a default value) else: sig.init(reftype, refvalue) typecheck(sig, default) sig.reset() # The general case else: typecheck(sig, default) # Does it include an xargs (*) argument i = lenargs if info["xname"]: # Get type from signature if type(argsig) is dict: try: sig = argsig[info["xname"]] except KeyError: raise SignatureError("No xargs with name %s in %s" % ( info["xname"], repr(argsig))) else: try: sig = argsig[i] except IndexError: raise SignatureError("No xargs in signature %s" % ( repr(argsig),)) # If the implementation has an xargs, then the signature should try: if sig != xargs: raise SignatureError("%s is not %s" % (repr(sig), repr(xargs))) except IndexError: raise SignatureError("No xargs %s in signature %s" % ( info["xname"], repr(argsig))) i += 1 # Does it include an xxkw (**) argument if info["xxname"]: # Get type from signature if type(argsig) is dict: try: sig = argsig[info["xxname"]] except KeyError: raise SignatureError("No xxkw with name %s in %s" % ( info["xxname"], repr(argsig))) else: try: sig = argsig[i] except IndexError: raise SignatureError("No xxkw in signature %s" % ( repr(argsig),)) # If the implementation has an xxkw, the the signature should try: if sig != xxkw: raise SignatureError("%s is not %s" % (repr(sig), repr(xxkw))) except IndexError: raise SignatureError("No xxkw %s in signature %s" % ( info["xxname"], repr(argsig))) # Return info about function return info def _initArg(args, argsig): """Initialization needed for argument referencing An argument reference (see `arg` and `isarg`) needs information about the type of the argument it is refering to at call time. The trick is to make the type specification of the argument it is refering to into a `_ref` type. The `_ref` type has a reference to the argument that refered to this argument (the referee), and when it is type checked it will initialize its referee with the information about its type (second level initialization). args -- arguments argsig -- signature """ # Should I add a check of the refered argument against the mapping # keys in `isarg`? Should this be done in `checkfunc`? Do I need # a way to specify all other cases (not found as keys in the # mapping)? Is actually `checkfunc` and `initArg` one function # (since `checkfunc` expect `initArg` to be performed before it is # done - the stuff returned by `initArg` can be inserted in the # `info` dictionary of the function)? Will this make the # implementation of checkfunc/initArg easier? It will make it # possible to use `checkfunc` on functions without signatures, I # think. # And then to the problem of arg/isarg refering to arguments where # the type is known at definition time. It will then be more # efficient to replace arg/isarg with the actual type (and skip # the _ref trick, and not add arg/isarg to the list returned by # `initArg`. # OK, rewrite `checkfunc` to include `initArg` and include the # functionality described above (currently I'm not to sure about # the quality of the arg/isarg code in these two functions - to # implementation feels more complex than needed). # Also make a copy (and a list, when tuple) of the argsig, and # update the copy (and include it in info). Remember to update # `signature` accordingly # Search for `arg` and `isarg` specification in list and tuples if type(args) is tuple or type(args) is list: rest = [] for a in args: rest += _initArg(a, argsig) return rest # Search for `arg` and `isarg` specification in dictionaries (also key) elif type(args) is dict: rest = [] for k, v in args.items(): rest += _initArg(k, argsig) + _initArg(v, argsig) return rest # Have we found a `arg` or `isarg` specification? elif isinstance(args, arg): # It should have a number or name if not hasattr(args, "argref"): raise SignatureError("Refered argument number or name missing") # Named arguments use names if type(argsig) is dict and type(args.argref) is str: # Replace type specification of the refered arguments with `_ref` try: argsig[args.argref] = _ref(argsig[args.argref], args) except KeyError: raise SignatureError( "Refered argument %s not found" % (args.argref,)) # Return a list of `arg` and `argsig` type specifications if isinstance(argsig[args.argref], _ref): return [args] # Currently the argument tuple is represented as a list (mutable) elif type(argsig) is list and type(args.argref) is int: # Replace type specification of the refered arguments with `_ref` try: argsig[args.argref - 1] = _ref(argsig[args.argref - 1], args) except IndexError: raise SignatureError( "Refered argument %d not found" % (args.argref,)) # Return a list of `arg` and `argsig` type specifications if isinstance(argsig[args.argref - 1], _ref): return [args] # No match with `argsig` else: raise SignatureError("Refered argument %s not found in %s" % ( repr(args.argref), repr(argsig))) # None found return [] def _replacefunc(f, argspecs): """Replace function implementation Run time type-checking is performed by the function replacing the original function `f`. So when the user is calling `f` she is actually calling the function `sf` below. This function replaces `f` with `sf`. f -- function argspecs -- arguments specification """ # Replacement for f def sf(*argv, **kw): # Should type checking be performed? if not sf.__callTimeTC__: return f(*argv, **kw) # Check provided arguments (call-time) against signature try: checkargs(sf, argv, kw) except SignatureError as msg: raise ArgumentError(msg) # Perform actual method call try: result = f(*argv, **kw) except Exception: exctype, excmsg = sys.exc_info()[:2] if (sf.__signature__ == None or sf.__signature__[2] == None or exctype in sf.__signature__[2]): raise exctype(excmsg) else: raise ExceptionError("Unknown exception %s: %s" % ( repr(exctype), excmsg)) # Check return value against signature try: typecheck(sf.__signature__[1], result) except SignatureError as msg: raise ReturnValueError(msg) # Reset `arg` and `argsig` specifications for a in argspecs: a.reset() # Return result return result # Save attributes sf.__dict__.update(f.__dict__) # Give method its original name, documentation string, and module sf.__name__ = f.__name__ sf.__doc__ = f.__doc__ sf.__module__ = f.__module__ # Return the replacement function return sf def _addsig(f, args, ret, exc): """Add signature to function This functions does the actual work of adding a signature to function `f`. It checks if the implementation fullfills the signature (if possible). It also checks the arguments and return value at call time against the signature (argument types and return value type). Be aware that the current implementation asumes that the first argument in the definition of a bound method is always named `self`. It also asumes that all other functions never have a first argument named `self`. f -- function args -- argument signature ret -- return type exc -- expected exceptions """ # Have annotations been added to signature (always happens first time) if not hasattr(f, "__signature__"): # Arguments and return value specified by annontations if hasattr(f, "__func__"): annotations = f.__func__.__annotations__ else: annotations = f.__annotations__ if annotations: if (args == None and ret == None): args = annotations.copy() if 'return' in args: ret = args['return'] del args['return'] else: raise SignatureError("Signature specification twice") # If no annotations and no arguments in signture elif args == None: args = () # Create a copy of `args` used as initial version of `compargs` compargs = None if type(args) is tuple: compargs = list(args) # Elements are changed in `_initArg` elif type(args) is dict: compargs = args.copy() # Prepare second level initialization of `arg` and `isarg` argspecs = [] if compargs != None: argspecs = _initArg(args, compargs) + _initArg(ret, compargs) # Arguments are either a dictionary or a tuple (not list) if type(compargs) is list: compargs = tuple(compargs) # Has it been replaced? if hasattr(f, "__signature__"): siglist = list(f.__signature__) sf = f # Not? Do it else: # Replace function sf = _replacefunc(f, argspecs) siglist = [None, None, None] # Check f implementation against signature info = checkfunc(f, args) # Save argument specification if not allready provided if siglist[0] == None: siglist[0] = args else: raise SignatureError("Arguments types allready applied to function") # These details are saved to make the checkargs implementation effective if not hasattr(sf, "__funcdetails__"): if info["xname"] or info["xxname"]: argsig = _stripXsig(info["xname"], info["xxname"], compargs) else: argsig = compargs sf.__dict__["__funcdetails__"] = info sf.__dict__["__funcdetails__"]["argsig"] = argsig else: raise SignatureError("Function details allready added to function") # Return value specification provided (either signature or annontations) if ret != None: # Save argument specification if not allready provided if siglist[1] == None: siglist[1] = ret else: raise SignatureError("Return types allready applied to function") # Exception specification provided if exc != None: if siglist[2] == None: siglist[2] = exc else: raise SignatureError( "Exception types allready applied to function") # Generators are special: call time gives a generator (ret), and # the returned generator has a `__next__` function return one # result each time (this is the return value spec) if info["generator"]: siglist[1] = GeneratorType # Then we should replace the __next__ function of f, but since # the return value can reference thearguments I might wait to # actually implement this (see also the comments in the # _initArg function). # Save the new signature sf.__dict__["__signature__"] = tuple(siglist) sf.__dict__["__callTimeTC__"] = callTimeTC # The replacement for f return sf def signature(args=None, ret=None, exc=None): """Apply signature to a function This is a decorator for functions or methods. It applies the given signature to the function or the method. It checks if the implementation fullfills the signature (if possible). It also checks the arguments and return value at call time against the signature (argument types and return value type). Be aware that the current implementation asumes that the first argument in the definition of a bound method is always named `self`. It also asumes that all other functions never have a first argument named `self`. A `SignatureError` is raised if there is a type missmatch when the function/method is defined or called. args -- argument signature ret -- return type exc -- expected exceptions """ # Called when function is defined def addsig(f): if ret == None and exc == None and isinstance(args, Sig): return _addsig(f, **args) else: return _addsig(f, args, ret, exc) # Signature decorator with arguments used (without parantheses) if type(args) in [FunctionType, MethodType]: return _addsig(args, None, None, None) # The function called when f is defined else: return addsig class Sig(dict): """A class for signatures This class is used to create signature instances. You can use a disctionary, but this will only allow valid keys and is a nice way of saving reusable signatures. """ def __init__(self, args=None, ret=None, exc=None): """Initialize a signature Provide arguments, return value and exceptions when a new instance of a signature is created. args -- argument signature ret -- return type exc -- expected exceptions """ self.update(args=args, ret=ret, exc=exc) def __setitem__(self, key, value): """Set the value of one item in the signature Only arguments, return value, and exception are possible items. key -- name, either 'args', 'ret', or 'exc' value -- the sub-signature for this part of the complete signature """ if key in [args, ret, exc]: super(Sig, self).__setitem__(key, value) else: raise SignatureError( "Only valid keys are \"args\", \"ret\", and \"exc\"")