While `tlsfuzzer` is currently more of a framework than a full-featured tool, it still is possible to run tests against common server configurations with a little bit of effort. Simple workflow =============== Preparation ----------- To run the scripts you will need 3 libraries: * [six](https://github.com/benjaminp/six) ([PyPI](https://pypi.python.org/pypi/six)) * [ecdsa](https://github.com/warner/python-ecdsa) ([PyPI](https://pypi.python.org/pypi/ecdsa)) * [tlslite-ng](https://github.com/tlsfuzzer/tlslite-ng) ([PyPI](https://pypi.python.org/pypi/tlslite-ng)) It's common that `six` is already installed, or is available from the operating system repository. Alternatively it can be installed by running `pip install six` as root. The other two can be installed (again, using `pip install ecdsa` and `pip install tlslite-ng`) or they all can be downloaded to a single location to minimise dependence on root privileges and permanent changes to the system. For the rest of this tutorial, I'll follow the latter approach. **Note:** the above libraries and `tlsfuzzer` support both Python 2 and Python 3, but they require at least Python 2.6. If you are running a modern distribution (RHEL 6 or later), using the provided `python` in the below commands is sufficient, if you're running older distribution, you will need to install new python, and use it for the below commands, usually switching `python` to `python26` (or similar). In other words, if you have `six` already installed, the environment can be prepared by running the following commands: ``` git clone https://github.com/tlsfuzzer/tlsfuzzer.git cd tlsfuzzer git clone https://github.com/warner/python-ecdsa .python-ecdsa ln -s .python-ecdsa/src/ecdsa/ ecdsa git clone https://github.com/tlsfuzzer/tlslite-ng .tlslite-ng ln -s .tlslite-ng/tlslite/ tlslite ``` Running tests ============= When all dependencies are downloaded or installed, place yourself in the root directory of the project (one with `tlsfuzzer`, `tests` and `scripts` directories) and you can start running tests. All tests support a minimum set of parameters: * `-h` to specify the hostname of the server under test (tests usually default to `localhost`) * `-p` to specify the port of the server under tests (tests default to 4433) * `--help` to display all options supported by a given script * names of the scenarios that are to be run (if not provided, all tests in a script are run) so to test if a server running on `example.com` on port 433 is not vulnerable to the [Bleichenbacher attack](http://archiv.infsec.ethz.ch/education/fs08/secsem/bleichenbacher98.pdf), you need to run the following command: ``` PYTHONPATH=. python scripts/test-bleichenbacher-workaround.py -h example.com -p 443 ``` A run of it will look something like this: ``` (beginning omitted) zero byte in last byte of random padding ... OK zero byte in first byte of random padding ... OK Test end successful: 8 failed: 0 ``` That prints the names of tests being run (e.g. "zero byte in first byte of random padding") and the overall test result, that 8 tests were run and the server behaved as expected, and in 0 situations the test failed. **Note**: unless stated otherwise in the help message of a specific test case, the scripts usually expect a HTTP server with no client certificate authentication. In such case (where there were no errors), that usually ends the testing and identifies the server as following the relevant RFC documents (like [RFC 5246](https://tools.ietf.org/html/rfc5246)) or not vulnerable to the vulnerability. Investigating errors ==================== Theory of operation ------------------- To be able to read the error messages of `tlsfuzzer` scripts, it's necessary to know a little about how it works internally. The simplest test script is the [test-conversation.py](https://github.com/tlsfuzzer/tlsfuzzer/blob/master/scripts/test-conversation.py), in it you will find a lot of boilerplate, and a single test scenario: ``` conversation = Connect(host, port) node = conversation ciphers = [CipherSuite.TLS_RSA_WITH_AES_128_CBC_SHA, CipherSuite.TLS_EMPTY_RENEGOTIATION_INFO_SCSV] node = node.add_child(ClientHelloGenerator(ciphers)) node = node.add_child(ExpectServerHello()) node = node.add_child(ExpectCertificate()) node = node.add_child(ExpectServerHelloDone()) node = node.add_child(ClientKeyExchangeGenerator()) node = node.add_child(ChangeCipherSpecGenerator()) node = node.add_child(FinishedGenerator()) node = node.add_child(ExpectChangeCipherSpec()) node = node.add_child(ExpectFinished()) node = node.add_child(ApplicationDataGenerator( bytearray(b"GET / HTTP/1.0\n\n"))) node = node.add_child(ExpectApplicationData()) node = node.add_child(AlertGenerator(AlertLevel.warning, AlertDescription.close_notify)) node = node.add_child(ExpectAlert()) node.next_sibling = ExpectClose() conversations["sanity"] = conversation ``` What this code does, is it sets up a list of things to do and things to expect from the server. In this case, it performs a simple [RSA key exchange](https://tools.ietf.org/html/rfc5246#page-36), an HTTP GET request and expects data back. Nodes with `Generator` in the name are creating messages and sending them to the server, nodes that have `Expect` in the name will pause execution (with a timeout) and wait for a message from the server. In some cases, multiple behaviours are acceptable, this is implemented using the "siblings". In the above example, after sending the `close_notify` Alert to server, the test case expects the server to either send an alert of its own (`ExpectAlert`), *or* to just close the connection (`ExpectClose`). If a node has no children, an implicit (TCP) connection close is placed there. Expect nodes not only verify that the messages received match the expected type, but also that it is matching other messages (e.g. that the Server Hello doesn't advertise support for extensions that the Client Hello didn't include). In other words, unless there are specific options set, tlsfuzzer will behave as a strict, well-behaved TLS client. General note ------------ Scripts expect specific *behaviour* from a server, not specific error or message. In other words, a *passing* test is a test to which the behaviour of the server matched the expectation (in some cases that may mean that server did report a failure through Alert message). A *failing* test is a test in which the server did not match expected behaviour - for example did not detect an error it should have detected and continued handshake. Reading error messages ---------------------- Typical error message looks like this: ``` zero byte in first byte of random padding ... Error encountered while processing node (child: ) with last message being: Error while processing Traceback (most recent call last): File "scripts/test-bleichenbacher-workaround.py", line 250, in main runner.run() File "/root/tlsfuzzer/tlsfuzzer/runner.py", line 178, in run node.process(self.state, msg) File "/root/tlsfuzzer/tlsfuzzer/expect.py", line 571, in process raise AssertionError(problem_desc) AssertionError: Expected alert description "bad_record_mac" does not match received "handshake_failure" ``` The first line, informs us which scenario was running when the error occurred, in this case "zero byte in first byte of random padding". Second line tells us which node was being processed, in this case it was `ExpectAlert` and its child is `ExpectClose`. Finally, the last line tells us what went wrong, in this case the error was that the test expected an alert with `bad_record_mac` but got `handshake_failure`. ### Common errors #### Alert description mismatch Pattern: ``` Error encountered while processing node