You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
53 lines
1.3 KiB
53 lines
1.3 KiB
4 years ago
|
Metadata-Version: 2.1
|
||
|
|
Name: untokenize
|
||
|
|
Version: 0.1.1
|
||
|
|
Summary: Transforms tokens into original source code (while preserving whitespace).
|
||
|
|
Home-page: https://github.com/myint/untokenize
|
||
|
|
Author: Steven Myint
|
||
|
|
Author-email: UNKNOWN
|
||
|
|
License: Expat License
|
||
|
|
Keywords: tokenize,untokenize,transform,generate
|
||
|
|
Platform: UNKNOWN
|
||
|
|
Classifier: Intended Audience :: Developers
|
||
|
|
Classifier: Environment :: Console
|
||
|
|
Classifier: Programming Language :: Python :: 2.6
|
||
|
|
Classifier: Programming Language :: Python :: 2.7
|
||
|
|
Classifier: Programming Language :: Python :: 3
|
||
|
|
Classifier: License :: OSI Approved :: MIT License
|
||
|
|
|
||
|
|
============
|
||
|
|
untokenize
|
||
|
|
============
|
||
|
|
|
||
|
|
*untokenize* transforms tokens into source code. Unlike the standard library's
|
||
|
|
``tokenize.untokenize()``, it preserves the original whitespace between tokens.
|
||
|
|
|
||
|
|
.. image:: https://travis-ci.org/myint/untokenize.png?branch=master
|
||
|
|
:target: https://travis-ci.org/myint/untokenize
|
||
|
|
:alt: Build status
|
||
|
|
|
||
|
|
|
||
|
|
Usage
|
||
|
|
=====
|
||
|
|
|
||
|
|
.. code-block:: python
|
||
|
|
|
||
|
|
import untokenize
|
||
|
|
source_code = untokenize.untokenize(tokens)
|
||
|
|
|
||
|
|
|
||
|
|
Tests
|
||
|
|
=====
|
||
|
|
|
||
|
|
To run the unit tests::
|
||
|
|
|
||
|
|
$ ./test_untokenize.py
|
||
|
|
|
||
|
|
There is also an acid test. It tokenizes Python code and confirms that the code
|
||
|
|
generated by untokenize exactly matches the original source code from before
|
||
|
|
tokenization::
|
||
|
|
|
||
|
|
$ ./test_acid.py
|
||
|
|
|
||
|
|
|