Rename integration tests to system tests, since they cover the entire system

*cough e82bdf4 *cough

CHANGELOG.rst

@@ -1,6 +1,11 @@
 Changelog
 =========
 
+2016-03-03
+----------
+Anders Ingemann:
+  * Rename integration tests to system tests
+
 2016-02-23
 ----------
 Nicolas Braud-Santoni:

bootstrapvz/remote/README.rst

@@ -14,7 +14,7 @@ when doing this multiple times SSHing into the machines and copying the
 manifests can be a hassle.
 
 Lastly, the main motivation for supporting remote bootstrapping is the
-automation of `integration testing <../../tests/integration>`__.
+automation of `system testing <../../tests/system>`__.
 As you will see `further down <#bootstrap-vz-remote>`__,
 bootstrap-vz is able to select which build server is required
 for a specific test and run the bootstrapping procedure on said server.
@@ -172,7 +172,7 @@ at ``/root/guest_additions.iso`` on server 1, while server 2 has it at
 Run settings
 ~~~~~~~~~~~~~~
 The run settings hold information about how to start a bootstrapped image.
-This is useful only when running integration tests.
+This is useful only when running system tests.
 
 .. code:: yaml
 

docs/conf.py

@@ -287,11 +287,11 @@ for readme_path in glob.glob('../bootstrapvz/plugins/*/README.rst'):
 			include.write('.. include:: ' + path_to_readme)
 
 
-for readme_path in glob.glob('../tests/integration/providers/*/README.rst'):
+for readme_path in glob.glob('../tests/system/providers/*/README.rst'):
 	provider_name = os.path.basename(os.path.dirname(readme_path))
-	include_path = os.path.join('testing/integration_test_providers', provider_name + '.rst')
+	include_path = os.path.join('testing/system_test_providers', provider_name + '.rst')
 	if not os.path.exists(include_path):
-		path_to_readme = os.path.join('../../../tests/integration/providers', provider_name, 'README.rst')
+		path_to_readme = os.path.join('../../../tests/system/providers', provider_name, 'README.rst')
 		with open(include_path, 'w') as include:
 			include.write('.. include:: ' + path_to_readme)
 

docs/testing/index.rst

@@ -6,7 +6,7 @@ Testing
    :hidden:
 
    unit_tests
-   integration_tests
+   system_tests
-   integration_test_providers/index
+   system_test_providers/index
 
 .. include:: ../../tests/README.rst

docs/testing/integration_test_providers/index.rst

@@ -1,10 +0,0 @@
-Integration test providers
-==========================
-
-.. toctree::
-   :maxdepth: 1
-   :glob:
-
-   *
-
-.. include:: ../../../tests/integration/providers/README.rst

docs/testing/integration_tests.rst

@@ -1 +0,0 @@
-.. include:: ../../tests/integration/README.rst

docs/testing/system_test_providers/index.rst

@@ -0,0 +1,10 @@
+System test providers
+=====================
+
+.. toctree::
+   :maxdepth: 1
+   :glob:
+
+   *
+
+.. include:: ../../../tests/system/providers/README.rst

docs/testing/system_tests.rst

@@ -0,0 +1 @@
+.. include:: ../../tests/system/README.rst

tests/integration/providers/docker/README.rst

@@ -1,7 +0,0 @@
-Docker
-------
-
-
-Dependencies
-~~~~~~~~~~~~
-The host machine running the integration tests must have docker installed.

tests/integration/providers/ec2/README.rst

@@ -1,7 +0,0 @@
-EC2
----
-
-Dependencies
-~~~~~~~~~~~~
-The host machine running the integration tests must have the python package
-``boto`` installed (``>= 2.14.0``).

tests/system/README.rst

@@ -1,11 +1,11 @@
-Integration tests
+System tests
-=================
+============
-`Integration tests`__ test
+`System tests`__ test
 bootstrap-vz in its entirety.
 This testing includes building images from manifests and
 creating/booting said images.
 
-__ http://en.wikipedia.org/wiki/Integration_testing
+__ http://en.wikipedia.org/wiki/System_testing
 
 Since hardcoding manifests for each test, bootstrapping them and booting the
 resulting images is too much code for a single test, a testing harness has
@@ -15,12 +15,12 @@ been developed that reduces each test to it's bare essentials:
 * Boot an instance from a manifest
 * Run tests on the booted instance
 
-In order for the integration testing harness to be able to bootstrap it must
+In order for the system testing harness to be able to bootstrap it must
 know about your `build-servers <../../bootstrapvz/remote#build-servers-yml>`__.
 Depending on the manifest that is bootstrapped, the harness chooses
 a fitting build-server, connects to it and starts the bootstrapping process.
 
-When running integration tests, the framework will look for ``build-servers.yml``
+When running system tests, the framework will look for ``build-servers.yml``
 at the root of the repo and raise an error if it is not found.
 
 
@@ -65,7 +65,7 @@ randomly generated on load.
 
 Missing parts
 -------------
-The integration testing harness is in no way complete.
+The system testing harness is in no way complete.
 
 * It still has no support for providers other than Virtualbox, EC2 and Docker.
 * Creating an SSH connection to a booted instance is cumbersome and does not

tests/system/providers/README.rst

@@ -1,4 +1,4 @@
-Integration testing providers are implemented on top of the abstraction
+System testing providers are implemented on top of the abstraction
 that is the testing harness.
 
 Implementation
@@ -21,7 +21,7 @@ resources, once testing is done (or failed, so remember to wrap ``yield`` in a
 
 Debugging
 ---------
-When developing an integration test provider, debugging through multiple
+When developing a system test provider, debugging through multiple
 invocations of ``tox`` can be cumbersome. A short test script, which sets
 up logging and invokes a specific test can be used instead:
 
@@ -32,7 +32,7 @@ Example:
 
     #!/usr/bin/env python
 
-    from tests.integration.docker_tests import test_stable
+    from tests.system.docker_tests import test_stable
     from bootstrapvz.base.main import setup_loggers
 
     setup_loggers({'--log': '-', '--color': 'default', '--debug': True})

tests/system/providers/docker/README.rst

@@ -0,0 +1,7 @@
+Docker
+------
+
+
+Dependencies
+~~~~~~~~~~~~
+The host machine running the system tests must have docker installed.

tests/system/providers/ec2/README.rst

@@ -0,0 +1,7 @@
+EC2
+---
+
+Dependencies
+~~~~~~~~~~~~
+The host machine running the system tests must have the python package
+``boto`` installed (``>= 2.14.0``).

tests/system/providers/ec2/__init__.py

@@ -1,5 +1,5 @@
 from contextlib import contextmanager
-from tests.integration.tools import waituntil
+from tests.system.tools import waituntil
 import logging
 log = logging.getLogger(__name__)
 

tests/system/providers/virtualbox/README.rst

@@ -5,5 +5,5 @@ Virtualbox
 Dependencies
 ~~~~~~~~~~~~
 VirtualBox itself is required on the machine that is
-running the integration tests. The same machine also needs to
+running the system tests. The same machine also needs to
 have python package ``vboxapi`` (``>=1.0``) installed.

tests/system/providers/virtualbox/instance.py

@@ -1,6 +1,6 @@
 import virtualbox
 from contextlib import contextmanager
-from tests.integration.tools import waituntil
+from tests.system.tools import waituntil
 import logging
 log = logging.getLogger(__name__)
 
@@ -59,7 +59,7 @@ class VirtualBoxInstance(object):
 	def boot(self):
 		log.debug('Booting vbox machine `{name}\''.format(name=self.name))
 		self.machine.launch_vm_process(self.session, 'headless').wait_for_completion(-1)
-		from tests.integration.tools import read_from_socket
+		from tests.system.tools import read_from_socket
 		# Gotta figure out a more reliable way to check when the system is done booting.
 		# Maybe bootstrapped unit test images should have a startup script that issues
 		# a callback to the host.

tests/system/tools/__init__.py

@@ -20,7 +20,7 @@ def boot_manifest(manifest_data, boot_vars={}):
 	manifest = Manifest(data=manifest_data)
 
 	import importlib
-	provider_module = importlib.import_module('tests.integration.providers.' + manifest.provider['name'])
+	provider_module = importlib.import_module('tests.system.providers.' + manifest.provider['name'])
 
 	prepare_bootstrap = getattr(provider_module, 'prepare_bootstrap', noop)
 	with prepare_bootstrap(manifest, build_server):

tox.ini

@@ -16,14 +16,14 @@ deps =
     nose-cov
 commands = nosetests --with-coverage --cover-package=bootstrapvz --cover-inclusive --verbose {posargs:tests/unit}
 
-[testenv:integration]
+[testenv:system]
 deps =
     nose
     nose-cov
     nose-htmloutput
     Pyro4 >= 4.30
     pyvbox >= 0.2.0
-commands = nosetests --with-coverage --cover-package=bootstrapvz --cover-inclusive --with-html --html-file=integration.html --verbose {posargs:tests/integration}
+commands = nosetests --with-coverage --cover-package=bootstrapvz --cover-inclusive --with-html --html-file=system.html --verbose {posargs:tests/system}
 
 [testenv:docs]
 changedir = docs