kevinheyer/bootstrap-vz
1
0
Fork 0
mirror of https://github.com/kevingruesser/bootstrap-vz.git synced 2025-10-10 17:19:51 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Merge branch 'full-sphinx-documentation' into development

Browse source
This commit is contained in:
Anders Ingemann 2015-04-12 15:14:23 +02:00
commit f1e98fdd8e
59 changed files with 1170 additions and 401 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

48
CHANGELOG
View file

@ -1,48 +0,0 @@
2014-11-23:
Noah Fontes:
* Add support for enhanced networking on EC2 images
2014-07-12:
Tiago Ilieve:
* Fixes #96: AddBackports is now a common task
2014-07-09:
Anders Ingemann:
* Allow passing data into the manifest
* Refactor logging setup to be more modular
* Convert every JSON file to YAML
* Convert "provider" into provider specific section
2014-07-02:
Vladimir Vitkov:
* Improve grub options to work better with virtual machines
2014-06-30:
Tomasz Rybak:
* Return information about created image
2014-06-22:
Victor Marmol:
* Enable the memory cgroup for the Docker plugin
2014-06-19:
Tiago Ilieve:
* Fixes #94: allow stable/oldstable as release name on manifest
Vladimir Vitkov:
* Improve ami listing performance
2014-06-07:
Tiago Ilieve:
* Download `gsutil` tarball to workspace instead of working directory
* Fixes #97: remove raw disk image created by GCE after build
2014-06-06:
Ilya Margolin:
* pip_install plugin
2014-05-23:
Tiago Ilieve:
* Fixes #95: check if the specified APT proxy server can be reached
2014-05-04:
Dhananjay Balan:
* Salt minion installation & configuration plugin
* Expose debootstrap --include-packages and --exclude-packages options to manifest
2014-05-03:
Anders Ingemann:
* Require hostname setting for vagrant plugin
* Fixes #14: S3 images can now be bootstrapped outside EC2.
* Added enable_agent option to puppet plugin
2014-05-02:
Tomasz Rybak:
* Added Google Compute Engine Provider

77
CHANGELOG.rst Normal file
View file

@ -0,0 +1,77 @@
Changelog
=========
2014-11-23:
Noah Fontes:
+ Add support for enhanced networking on EC2 images
2014-07-12:
Tiago Ilieve:
+ Fixes #96: AddBackports is now a common task
2014-07-09:
Anders Ingemann:
+ Allow passing data into the manifest
+ Refactor logging setup to be more modular
+ Convert every JSON file to YAML
+ Convert "provider" into provider specific section
2014-07-02:
Vladimir Vitkov:
+ Improve grub options to work better with virtual machines
2014-06-30:
Tomasz Rybak:
+ Return information about created image
2014-06-22:
Victor Marmol:
+ Enable the memory cgroup for the Docker plugin
2014-06-19:
Tiago Ilieve:
+ Fixes #94: allow stable/oldstable as release name on manifest
Vladimir Vitkov:
+ Improve ami listing performance
2014-06-07:
Tiago Ilieve:
+ Download `gsutil` tarball to workspace instead of working directory
+ Fixes #97: remove raw disk image created by GCE after build
2014-06-06:
Ilya Margolin:
+ pip_install plugin
2014-05-23:
Tiago Ilieve:
+ Fixes #95: check if the specified APT proxy server can be reached
2014-05-04:
Dhananjay Balan:
+ Salt minion installation & configuration plugin
+ Expose debootstrap --include-packages and --exclude-packages options to manifest
2014-05-03:
Anders Ingemann:
+ Require hostname setting for vagrant plugin
+ Fixes #14: S3 images can now be bootstrapped outside EC2.
+ Added enable_agent option to puppet plugin
2014-05-02:
Tomasz Rybak:
+ Added Google Compute Engine Provider

42
CONTRIBUTING.md
View file

@ -1,42 +0,0 @@
Contributing
============
Do you want to contribute to the bootstrap-vz project? Nice! Here is the basic workflow:
* Read the [development guidelines](http://bootstrap-vz.readthedocs.org/en/master/guidelines.html)
* Fork this repository.
* Make any changes you want/need.
* Check the coding style of your changes using [tox](http://tox.readthedocs.org/) by running `tox -e flake8`
and fix any warnings that may appear.
This check will be repeated by [Travis CI](https://travis-ci.org/andsens/bootstrap-vz)
once you send a pull request, so it's better if you check this beforehand.
* If the change is significant (e.g. a new plugin, manifest setting or security fix)
add your name and contribution to the [CHANGELOG](CHANGELOG).
* Commit your changes.
* Squash the commits if needed. For instance, it is fine if you have multiple commits describing atomic units
of work, but there's no reason to have many little commits just because of corrected typos.
* Push to your fork, preferably on a topic branch.
From here on there are two paths to consider:
If your patch is a new feature, e.g.: plugin, provider, etc. then:
* Send a pull request to the `development` branch. It will be merged into the `master` branch when we can make
sure that the code is stable.
If it is a bug/security fix:
* Send a pull request to the `master` branch.
--
Please try to be very descriptive about your changes when you write a pull request, stating what it does, why
it is needed, which use cases this change covers etc.
You may be asked to rebase your work on the current branch state, so it can be merged cleanly.
If you push a new commit to your pull request you will have to add a new comment to the PR,
provided that you want us notified. Github will otherwise not send a notification.
Be aware that your modifications need to be properly documented and pushed to the `gh-pages` branch, if they
concern anything done on `master`. Otherwise, they should be sent to the `gh-pages-dev`.
Happy hacking! :-)

55
docs/guidelines.rst → CONTRIBUTING.rst
View file

@ -1,6 +1,53 @@
Contributing
============
Sending pull requests
---------------------
Do you want to contribute to the bootstrap-vz project? Nice! Here is the basic workflow:
+ Read the `development guidelines <http://bootstrap-vz.readthedocs.org/en/master/contributing.html#development-guidelines>`__
+ Fork this repository.
+ Make any changes you want/need.
+ Check the coding style of your changes using `tox <http://tox.readthedocs.org/>`__ by running `tox -e flake8`
and fix any warnings that may appear.
This check will be repeated by `Travis CI <https://travis-ci.org/andsens/bootstrap-vz>`__
once you send a pull request, so it's better if you check this beforehand.
+ If the change is significant (e.g. a new plugin, manifest setting or security fix)
add your name and contribution to the `changelog <http://bootstrap-vz.readthedocs.org/en/master/changelog.html>`__.
+ Commit your changes.
+ Squash the commits if needed. For instance, it is fine if you have multiple commits describing atomic units
of work, but there's no reason to have many little commits just because of corrected typos.
+ Push to your fork, preferably on a topic branch.
From here on there are two paths to consider:
If your patch is a new feature, e.g.: plugin, provider, etc. then:
* Send a pull request to the `development` branch. It will be merged into the `master` branch when we can make
sure that the code is stable.
If it is a bug/security fix:
* Send a pull request to the `master` branch.
--
Please try to be very descriptive about your changes when you write a pull request, stating what it does, why
it is needed, which use cases this change covers etc.
You may be asked to rebase your work on the current branch state, so it can be merged cleanly.
If you push a new commit to your pull request you will have to add a new comment to the PR,
provided that you want us notified. Github will otherwise not send a notification.
Be aware that your modifications need to be properly documented and pushed to the `gh-pages` branch, if they
concern anything done on `master`. Otherwise, they should be sent to the `gh-pages-dev`.
Happy hacking! :-)
Development guidelines
======================
----------------------
The following guidelines should serve as general advice when
developing providers or plugins for bootstrap-vz. Keep in mind that
these guidelines are not rules , they are advice on how to better add
@ -14,7 +61,7 @@ value to the bootstrap-vz codebase.
This allows others to easily reproduce any
setup other people are running and makes it possible to share
manifests. `The official debian EC2 images <https:/aws.amazon.com/marketplace/seller-
profile?id=890be55d-32d8-4bc8-9042-2b4fd83064d5>`_
profile?id=890be55d-32d8-4bc8-9042-2b4fd83064d5>`__
for example can be reproduced using the manifests available
in the manifest directory of bootstrap-vz.
@ -60,7 +107,7 @@ value to the bootstrap-vz codebase.
This allows other tasks to interleave with the control-flow and add extended
functionality (e.g. because volume creation and mounting are two
separate tasks, `the prebootstrapped plugin
<http://andsens.github.io/bootstrap-vz/plugins.html#prebootstrapped>`_
<http://bootstrap-vz.readthedocs.org/en/master/plugins/prebootstrapped.html>`__
can replace the volume creation task with a task of its own that
creates a volume from a snapshot instead, but still reuse the mount task).
@ -105,7 +152,7 @@ guidelines. There however a few exceptions:
+ Ignore ``W191``: Indent with tabs not spaces
The codebase can be checked for any violations quite easily, since those rules are already specified in the
`tox <http://tox.readthedocs.org/>`_ configuration file.
`tox <http://tox.readthedocs.org/>`__ configuration file.
::
tox -e flake8

45
README.md
View file

@ -1,45 +0,0 @@
bootstrap-vz
===========================================
bootstrap-vz is a bootstrapping framework for Debian.
It is is specifically targeted at bootstrapping systems for virtualized environments.
bootstrap-vz runs without any user intervention and generates ready-to-boot images for
[a number of virtualization platforms](http://andsens.github.io/bootstrap-vz/providers.html).
Its aim is to provide a reproducable bootstrapping process using [manifests](http://andsens.github.io/bootstrap-vz/manifest.html) as well as supporting a high degree of customizability through plugins.
bootstrap-vz was coded from scratch in python once the bash script architecture that was used in the
[build-debian-cloud](https://github.com/andsens/build-debian-cloud) bootstrapper reached its
limits.
Documentation
-------------
The end-user documentation for bootstrap-vz is available
at [andsens.github.io/bootstrap-vz](http://andsens.github.io/bootstrap-vz).
There, you can discover [what the dependencies](http://andsens.github.io/bootstrap-vz/#dependencies)
for a specific cloud provider are, [see a list of available plugins](http://andsens.github.io/bootstrap-vz/plugins.html)
and learn [how you create a manifest](http://andsens.github.io/bootstrap-vz/manifest.html).
Installation
------------
bootstrap-vz has a master branch for stable releases and a development for, well, development.
After checking out the branch of your choice you can install the python dependencies by running
`python setup.py install`. However, depending on what kind of image you'd like to bootstrap,
there are other debian package dependencies as well, at the very least you will need `debootstrap`.
[The documentation](http://andsens.github.io/bootstrap-vz/) explains this in more detail.
Note that bootstrap-vz will tell you which tools it requires when they aren't
present (the different packages are mentioned in the error message), so you can
simply run bootstrap-vz once to get a list of the packages, install them,
and then re-run.
Developers
----------
The API documentation, development guidelines and an explanation of bootstrap-vz internals
can be found at [bootstrap-vz.readthedocs.org](http://bootstrap-vz.readthedocs.org).
Contributing
------------
Contribution guidelines are described on the [CONTRIBUTING](CONTRIBUTING.md) file. There's also a
[topic on the documentation](http://bootstrap-vz.readthedocs.org/en/development/guidelines.html#coding-style)
regarding the coding style.

148
README.rst Normal file
View file

@ -0,0 +1,148 @@
Introduction
============
bootstrap-vz is a bootstrapping framework for Debian.
It is is specifically targeted at bootstrapping systems for virtualized environments.
bootstrap-vz runs without any user intervention and
generates ready-to-boot images for a number of virtualization
platforms.
Its aim is to provide a reproducable bootstrapping process using
`manifests <http://bootstrap-vz.readthedocs.org/en/master/manifest.html>`__
as well as supporting a high degree of customizability through plugins.
bootstrap-vz was coded from scratch in python once the bash script
architecture that was used in the
`build-debian-cloud <https://github.com/andsens/build-debian-cloud>`__
bootstrapper reached its limits.
Documentation
-------------
The documentation for bootstrap-vz is available at
`bootstrap-vz.readthedocs.org <http://bootstrap-vz.readthedocs.org/en/master>`__.
There, you can discover `what the
dependencies <http://bootstrap-vz.readthedocs.org/en/master/index.html#dependencies>`__
for a specific cloud provider are, `see a list of available
plugins <http://bootstrap-vz.readthedocs.org/en/master/plugins>`__ and
learn `how you create a
manifest <http://bootstrap-vz.readthedocs.org/en/master/manifest.html>`__.
Installation
------------
bootstrap-vz has a master branch for stable releases and a development
for, well, development.
After checking out the branch of your choice you can install the
python dependencies by running ``python setup.py install``. However,
depending on what kind of image you'd like to bootstrap, there are
other debian package dependencies as well, at the very least you will
need ``debootstrap``.
`The documentation <http://bootstrap-vz.readthedocs.org/en/master>`__
explains this in more detail.
Note that bootstrap-vz will tell you which tools it requires when they
aren't present (the different packages are mentioned in the error
message), so you can simply run bootstrap-vz once to get a list of the
packages, install them, and then re-run.
Quick start
-----------
Here are a few quickstart tutorials for the most common images.
If you plan on partitioning your volume, you will need the ``parted``
package and ``kpartx``:
.. code:: sh
root@host:~# apt-get install parted kpartx
VirtualBox Vagrant
~~~~~~~~~~~~~~~~~~
::
user@host:~$ sudo -i # become root
root@host:~# git clone https://github.com/andsens/bootstrap-vz.git # Clone the repo
root@host:~# apt-get install qemu-utils debootstrap python-pip # Install dependencies from aptitude
root@host:~# pip install termcolor jsonschema fysom docopt pyyaml # Install python dependencies
root@host:~# bootstrap-vz/bootstrap-vz bootstrap-vz/manifests/virtualbox-vagrant.manifest.yml
If you want to use the `minimize\_size <http://bootstrap-vz.readthedocs.org/en/master/plugins/minimize_size.html>`__
plugin, you will have to install the ``zerofree`` package and `VMWare
Workstation <https://my.vmware.com/web/vmware/info/slug/desktop_end_user_computing/vmware_workstation/10_0>`__
as well.
Amazon EC2 EBS backed AMI
~~~~~~~~~~~~~~~~~~~~~~~~~
.. code:: sh
user@host:~$ sudo -i # become root
root@host:~# git clone https://github.com/andsens/bootstrap-vz.git # Clone the repo
root@host:~# apt-get install debootstrap python-pip # Install dependencies from aptitude
root@host:~# pip install termcolor jsonschema fysom docopt pyyaml boto # Install python dependencies
root@host:~# bootstrap-vz/bootstrap-vz bootstrap-vz/manifests/ec2-ebs-debian-official-amd64-pvm.manifest.yml
To bootstrap S3 backed AMIs, bootstrap-vz will also need the
``euca2ools`` package. However, version 3.2.0 is required meaning you
must however install it directly from the eucalyptus repository like
this:
.. code:: sh
apt-get install --no-install-recommends python-dev libxml2-dev libxslt-dev gcc
pip install git+git://github.com/eucalyptus/euca2ools.git@v3.2.0
Cleanup
-------
bootstrap-vz tries very hard to clean up after itself both if a run was
successful but also if it failed. This ensures that you are not left
with volumes still attached to the host which are useless. If an error
occurred you can simply correct the problem that caused it and rerun
everything, there will be no leftovers from the previous run (as always
there are of course rare/unlikely exceptions to that rule). The error
messages should always give you a strong hint at what is wrong, if that
is not the case please consider `opening an
issue <https://github.com/andsens/bootstrap-vz/issues>`__ and attach
both the error message and your manifest (preferably as a gist or
similar).
Dependencies
------------
bootstrap-vz has a number of dependencies depending on the target
platform and `the selected plugins <http://bootstrap-vz.readthedocs.org/en/master/plugins>`__.
At a bare minimum the following python libraries are needed:
* `termcolor <https://pypi.python.org/pypi/termcolor>`__
* `fysom <https://pypi.python.org/pypi/fysom>`__
* `jsonschema <https://pypi.python.org/pypi/jsonschema>`__
* `docopt <https://pypi.python.org/pypi/docopt>`__
* `pyyaml <https://pypi.python.org/pypi/pyyaml>`__
To bootstrap Debian itself
`debootstrap <https://packages.debian.org/wheezy/debootstrap>`__
is needed as well.
Any other requirements are dependent upon the manifest configuration
and are detailed in the corresponding sections of the documentation.
bootstrap-vz will however warn you if a requirement has not been met,
before the bootstrapping process begins.
Developers
----------
The API documentation, development guidelines and an explanation of
bootstrap-vz internals can be found at
`bootstrap-vz.readthedocs.org <http://bootstrap-vz.readthedocs.org/en/master/developers>`__.
Contributing
------------
Contribution guidelines are described in the documentation under
`Contributing <http://bootstrap-vz.readthedocs.org/en/master/developers/contributing.html>`__.
There's also
`a topic <http://bootstrap-vz.readthedocs.org/en/master/developers/contributing.html#coding-style>`__
regarding the coding style.

1
bootstrapvz/base/fs/volume.py
View file

@ -70,6 +70,7 @@ class Volume(FSMProxy):
rather than a loopback device or a network block device.
:param _e_obj e: Event object containing arguments to create()
Keyword arguments to link_dm_node() are:
:param int logical_start_sector: The sector the volume should start at in the new volume

12
bootstrapvz/plugins/admin_user/README.rst Normal file
View file

@ -0,0 +1,12 @@
Admin user
----------
This plugin creates a user with passwordless sudo privileges. It also
disables the SSH root login. If the EC2 init scripts are installed, the
script for fetching the SSH authorized keys will be adjust to match the
username specified.
Settings
~~~~~~~~
- ``username``: The username of the account to create. *``required``*

21
bootstrapvz/plugins/apt_proxy/README.rst Normal file
View file

@ -0,0 +1,21 @@
APT Proxy
---------
This plugin creates a proxy configuration file for APT, so you could
enjoy the benefits of using cached packages instead of downloading them
from the mirror every time. You could just install ``apt-cacher-ng`` on
the host machine and then add ``"address": "127.0.0.1"`` and
``"port": 3142`` to the manifest file.
Settings
~~~~~~~~
- ``address``: The IP or host of the proxy server.
*``required``*
- ``port``: The port (integer) of the proxy server.
*``required``*
- ``persistent``: Whether the proxy configuration file should remain on
the machine or not.
Valid values: true, false
Default: ``false``
*``optional``*

23
bootstrapvz/plugins/cloud_init/README.rst Normal file
View file

@ -0,0 +1,23 @@
cloud-init
----------
This plugin installs and configures
`cloud-init <https://packages.debian.org/wheezy-backports/cloud-init>`__
on the system. Depending on the release it installs it from either
backports or the main repository.
cloud-init is only compatible with Debian wheezy and upwards.
Settings
~~~~~~~~
- ``username``: The username of the account to create.
*``required``*
- ``disable_modules``: A list of strings specifying which cloud-init
modules should be disabled.
*``optional``*
- ``metadata_sources``: A string that sets the
`datasources <http://cloudinit.readthedocs.org/en/latest/topics/datasources.html>`__
that cloud-init should try fetching metadata from. The source is
automatically set when using the ec2 provider.
*``optional``*

18
bootstrapvz/plugins/docker_daemon/README.rst Normal file
View file

@ -0,0 +1,18 @@
Docker daemon
-------------
Install `docker <http://www.docker.io/>`__ daemon in the image. Uses
init scripts for the official repository.
This plugin can only be used if the distribution being bootstrapped is
at least ``wheezy``, as Docker needs a kernel version ``3.8`` or higher,
which is available at the ``wheezy-backports`` repository. There's also
an architecture requirement, as it runs only on ``amd64``.
Settings
~~~~~~~~
- ``version``: Selects the docker version to install. To select the
latest version simply omit this setting.
Default: ``latest``
*``optional``*

19
bootstrapvz/plugins/image_commands/README.rst Normal file
View file

@ -0,0 +1,19 @@
Image commands
--------------
The image commands plugin allows you to run arbitrary commands during
the bootstrap process. The commands are run at an indeterminate point
*after* packages have been installed, but *before* the volume has been
unmounted.
Settings
~~~~~~~~
- ``commands``: A list of lists containing strings. Each top-level item
is a single command, while the strings inside each list comprise
parts of a command. This allows for proper shell argument escaping
(to circumvent this, simply put the entire command in a single
string). In addition to the manifest variables ``{root}`` is also
available. It points at the root of the image volume.
*``required``*
*``manifest vars``*

37
bootstrapvz/plugins/minimize_size/README.rst Normal file
View file

@ -0,0 +1,37 @@
minimize size
-------------
This plugin can be used to reduce the size of the resulting image. Often
virtual volumes are much smaller than their reported size until any data
is written to them. During the bootstrapping process temporary data like
the aptitude cache is written to the volume only to be removed again.
The minimize size plugin employs three different strategies to keep a
low volume footprint:
- Mount folders from the host into key locations of the image volume to
avoid any unneccesary disk writes.
- Use `zerofree <http://intgat.tigress.co.uk/rmy/uml/index.html>`__ to
deallocate unused sectors on the volume. On an unpartitioned volume
this will be done for the entire volume, while it will only happen on
the root partition for partitioned volumes.
- Use
`vmware-vdiskmanager <https://www.vmware.com/support/ws45/doc/disks_vdiskmanager_eg_ws.html>`__
to shrink the real volume size (only applicable when using vmdk
backing). The tool is part of the `VMWare
Workstation <https://my.vmware.com/web/vmware/info/slug/desktop_end_user_computing/vmware_workstation/10_0>`__
package.
Settings
~~~~~~~~
- ``zerofree``: Specifies if it should mark unallocated blocks as
zeroes, so the volume could be better shrunk after this.
Valid values: true, false
Default: false
*``optional``*
- ``shrink``: Whether the volume should be shrunk. This setting works
best in conjunction with the zerofree tool.
Valid values: true, false
Default: false
*``optional``*

12
bootstrapvz/plugins/ntp/README.rst Normal file
View file

@ -0,0 +1,12 @@
NTP
---
This plugins installs the Network Time Protocol daemon and optionally
defines which time servers it should use.
Settings
~~~~~~~~
- ``servers``: A list of strings specifying which servers should be
used to synchronize the machine clock.
*``optional``*

13
bootstrapvz/plugins/opennebula/README.md
View file

@ -1,13 +0,0 @@
# Open Nebula provider
This provider adds OpenNebula contextualization to the virtual image (see http://opennebula.org/documentation:rel4.2:cong).
It set ups the network and ssh keys. TO do so you should configure your virtual machine context with something like:
ETH0_DNS $NETWORK[DNS, NETWORK_ID=2]
ETH0_GATEWAY $NETWORK[GATEWAY, NETWORK_ID=2]
ETH0_IP $NIC[IP, NETWORK_ID=2]
ETH0_MASK $NETWORK[MASK, NETWORK_ID=2]
ETH0_NETWORK $NETWORK[NETWORK, NETWORK_ID=2]
FILES path_to_my_ssh_public_key.pub

27
bootstrapvz/plugins/opennebula/README.rst Normal file
View file

@ -0,0 +1,27 @@
Open Nebula
-----------
This plugin adds `OpenNebula
contextualization <http://opennebula.org/documentation:rel4.2:cong>`__
to the image, which sets up the network configuration and SSH keys.
The virtual machine context should be configured as follows:
::
ETH0_DNS $NETWORK[DNS, NETWORK_ID=2]
ETH0_GATEWAY $NETWORK[GATEWAY, NETWORK_ID=2]
ETH0_IP $NIC[IP, NETWORK_ID=2]
ETH0_MASK $NETWORK[MASK, NETWORK_ID=2]
ETH0_NETWORK $NETWORK[NETWORK, NETWORK_ID=2]
FILES path_to_my_ssh_public_key.pub
The plugin will install all *.pub* files in the root authorized\_keys
file. When using the ec2 provider, the USER\_EC2\_DATA will be executed
if present.
Settings
~~~~~~~~
This plugin has no settings. To enable it add ``"opennebula":{}`` to the
plugin section of the manifest.

14
bootstrapvz/plugins/pip_install/README.rst Normal file
View file

@ -0,0 +1,14 @@
Pip install
-----------
Install packages from the Python Package Index via pip.
Installs ``build-essential`` and ``python-dev`` debian packages, so
Python extension modules can be built.
Settings
~~~~~~~~
- ``packages``: Python packages to install, a list of strings. The list
can contain anything that ``pip install`` would accept as an
argument, for example ``awscli==1.3.13``.

26
bootstrapvz/plugins/prebootstrapped/README.rst Normal file
View file

@ -0,0 +1,26 @@
prebootstrapped
---------------
When developing for bootstrap-vz, testing can be quite tedious since the
bootstrapping process can take a while. The prebootstrapped plugin
solves that problem by creating a snapshot of your volume right after
all the software has been installed. The next time bootstrap-vz is run,
the plugin replaces all volume preparation and bootstrapping tasks and
recreates the volume from the snapshot instead.
The plugin assumes that the users knows what he is doing (e.g. it
doesn't check whether bootstrap-vz is being run with a partitioned
volume configuration, while the snapshot is unpartitioned).
When no snapshot or image is specified the plugin creates one and
outputs its ID/path. Specifying an ID/path enables the second mode of
operation which recreates the volume from the specified snapshot instead
of creating it from scratch.
Settings
~~~~~~~~
- ``snapshot``: ID of the EBS snapshot to use. This setting only works
with EBS backed EC2 configurations.
- ``image``: Path to the loopbackvolume snapshot. This setting works
with all configurable volume backings except EBS.

24
bootstrapvz/plugins/puppet/README.rst Normal file
View file

@ -0,0 +1,24 @@
Puppet
------
Installs `puppet <http://puppetlabs.com/>`__ and optionally applies a
manifest inside the chroot. You can also have it copy your puppet
configuration into the image so it is readily available once the image
is booted.
Keep in mind that when applying a manifest, the system is in a chrooted
environment. This can prevent daemons from running properly (e.g.
listening to ports), they will also need to be shut down gracefully
(which bootstrap-vz cannot do) before unmounting the volume. It is
advisable to avoid starting any daemons inside the chroot at all.
Settings
~~~~~~~~
- ``manifest``: Path to the puppet manifest that should be applied.
*``optional``*
- ``assets``: Path to puppet assets. The contents will be copied into
``/etc/puppet`` on the image. Any existing files will be overwritten.
*``optional``*
- ``enable_agent``: Whether the puppet agent daemon should be enabled.
*``optional``*

11
bootstrapvz/plugins/root_password/README.rst Normal file
View file

@ -0,0 +1,11 @@
root password
-------------
Sets the root password. This plugin removes the task that disables the
SSH password authentication.
Settings
~~~~~~~~
- ``password``: The password for the root user.
*``required``*

26
bootstrapvz/plugins/salt/README.rst Normal file
View file

@ -0,0 +1,26 @@
Salt
----
Install `salt <http://www.saltstack.com/>`__ minion in the image. Uses
`salt-bootstrap <https://github.com/saltstack/salt-bootstrap>`__ script
to install.
Settings
~~~~~~~~
- ``install_source``: Source to install salt codebase from. ``stable``
for current stable, ``daily`` for installing the daily build, and
``git`` to install from git repository.
*``required``*
- ``version``: Only needed if you are installing from ``git``.
\ ``develop`` to install current development head, or provide any tag
name or commit hash from `salt
repo <https://github.com/saltstack/salt>`__
*``optional``*
- ``master``: Salt master FQDN or IP
*``optional``*
- ``grains``: Set `salt
grains <http://docs.saltstack.com/en/latest/topics/targeting/grains.html>`__
for this minion. Accepts a map with grain name as key and the grain
data as value.
*``optional``*

18
bootstrapvz/plugins/unattended_upgrades/README.rst Normal file
View file

@ -0,0 +1,18 @@
Unattended upgrades
-------------------
Enables the `unattended update/upgrade
feature <https://packages.debian.org/wheezy/unattended-upgrades>`__ in
aptitude. Enable it to have your system automatically download and
install security updates automatically with a set interval.
Settings
~~~~~~~~
- ``update_interval``: Days between running ``apt-get update``.
*``required``*
- ``download_interval``: Days between running
``apt-get upgrade --download-only``
*``required``*
- ``upgrade_interval``: Days between installing any security upgrades.
*``required``*

12
bootstrapvz/plugins/vagrant/README.rst Normal file
View file

@ -0,0 +1,12 @@
Vagrant
-------
Vagrant is a tool to quickly create virtualized environments. It uses
"boxes" to make downloading and sharing those environments easier. A box
is a tarball containing a virtual volumes accompanied by an `OVF
specification <http://en.wikipedia.org/wiki/Open_Virtualization_Format>`__
of the virtual machine.
This plugin creates a vagrant box that is ready to be shared or
deployed. At the moment it is only compatible with the VirtualBox
provider and doesn't requires any additional settings.

28
bootstrapvz/providers/azure/README.md → bootstrapvz/providers/azure/README.rst
View file

@ -1,22 +1,28 @@
Azure provider
===========
Azure
=====
This provider generates raw images for Microsoft Azure computing platform.
This provider generates raw images for Microsoft Azure computing
platform.
Setup
=====
-----
qemu-img >= 1.7.0 required to convert raw image to vhd fixed size disk.
This release is available in wheezy-backports.
*wget* must be installed on local computer.
Manifest must use the *raw* format, provider will automatically transform the disk to a vhd disk format.
Manifest must use the *raw* format, provider will automatically
transform the disk to a vhd disk format.
Do not create swap space on the OS disk:
The Windows Azure Linux Agent can automatically configure swap space using the local resource disk that is attached to the VM after provisioning on Azure. Modify the following parameters in /etc/waagent.conf appropriately:
The Windows Azure Linux Agent can automatically configure swap space
using the local resource disk that is attached to the VM after
provisioning on Azure. Modify the following parameters in
/etc/waagent.conf appropriately:
::
ResourceDisk.Format=y
ResourceDisk.Filesystem=ext4
@ -24,7 +30,10 @@ The Windows Azure Linux Agent can automatically configure swap space using the l
ResourceDisk.EnableSwap=y
ResourceDisk.SwapSizeMB=2048 ## NOTE: set this to whatever you need it to be.
You can specify a waagent.conf file to replace the default one in the manifest in the azure/waagent section of the provider:
You can specify a waagent.conf file to replace the default one in the
manifest in the azure/waagent section of the provider:
::
"system" : {
"waagent" : {
@ -33,4 +42,5 @@ You can specify a waagent.conf file to replace the default one in the manifest i
}
}, ...
Waagent versions are available at: https://github.com/Azure/WALinuxAgent/releases
Waagent versions are available at:
https://github.com/Azure/WALinuxAgent/releases

71
bootstrapvz/providers/ec2/README.rst Normal file
View file

@ -0,0 +1,71 @@
EC2
===
The `EC2 <http://aws.amazon.com/ec2/>`__ provider automatically creates
a volume for bootstrapping (be it EBS or S3), makes a snapshot of it
once it is done and registers it as an AMI. EBS volume backing only
works on an EC2 host while S3 backed volumes *should* work locally (at
this time however they do not, a fix is in the works).
Unless `the cloud-init plugin <http://bootstrap-vz.readthedocs.org/en/master/plugins/cloud_init.html>`__
is used, special startup scripts will be installed that automatically fetch the
configured authorized\_key from the instance metadata and save or run
any userdata supplied (if the userdata begins with ``#!`` it will be
run).
Credentials
-----------
The AWS credentials can be configured in two ways: Via the manifest or
through environment variables. To bootstrap S3 backed instances you will
need a user certificate and a private key in addition to the access key
and secret key, which are needed for bootstraping EBS backed instances.
The settings describes below should be placed in the ``credentials`` key
under the ``provider`` section.
- **``access-key``**: AWS access-key.
May also be supplied via the environment variable
``$AWS_ACCESS_KEY``
*``required for EBS & S3 backing``*
- **``secret-key``**: AWS secret-key.
May also be supplied via the environment variable
``$AWS_SECRET_KEY``
*``required for EBS & S3 backing``*
- **``certificate``**: Path to the AWS user certificate. Used for
uploading the image to an S3 bucket.
May also be supplied via the environment variable
``$AWS_CERTIFICATE``
*``required for S3 backing``*
- **``private-key``**: Path to the AWS private key. Used for uploading
the image to an S3 bucket.
May also be supplied via the environment variable
``$AWS_PRIVATE_KEY``
*``required for S3 backing``*
- **``user-id``**: AWS user ID. Used for uploading the image to an S3
bucket.
May also be supplied via the environment variable ``$AWS_USER_ID``
*``required for S3 backing``*
Example:
.. code:: yaml
---
provider:
name: ec2
virtualization: hvm
enhanced_networking: simple
credentials:
access-key: AFAKEACCESSKEYFORAWS
secret-key: thes3cr3tkeyf0ryourawsaccount/FS4d8Qdva
Dependencies
------------
To communicate with the AWS API `boto <https://github.com/boto/boto>`__
is required (version 2.14.0 or higher) you can install boto with
``pip install boto`` (on wheezy, the packaged version is too low). S3
images are chopped up and uploaded using
`euca2ools <https://github.com/eucalyptus/euca2ools>`__ (install with
``apt-get install euca2ools``).

9
bootstrapvz/providers/gce/README.rst Normal file
View file

@ -0,0 +1,9 @@
Google Compute Engine
---------------------
The `GCE <https://cloud.google.com/products/compute-engine/>`__ provider
can creates image as expected by GCE - i.e. raw disk image in \*.tar.gz
file. It can upload created images to Google Storage Engine (to URI
provided in manifest by ``gcs_destination``) and can register image to
be used by Google Compute Engine to project provided in manifest by
``gce_project``. Both of those functionalities are not fully tested yet.

8
bootstrapvz/providers/kvm/README.rst Normal file
View file

@ -0,0 +1,8 @@
KVM
---
The `KVM <http://www.linux-kvm.org/page/Main_Page>`__ provider creates
virtual images for Linux Kernel-based Virtual Machines. It supports the
installation of `virtio kernel
modules <http://www.linux-kvm.org/page/Virtio>`__ (paravirtualized
drivers for IO operations).

12
bootstrapvz/providers/virtualbox/README.rst Normal file
View file

@ -0,0 +1,12 @@
VirtualBox
----------
The `VirtualBox <https://www.virtualbox.org/>`__ provider can bootstrap
to both .vdi and .vmdk images (raw images are also supported but do not
run in VirtualBox). It's advisable to always use vmdk images for
interoperability (e.g.
`OVF <http://en.wikipedia.org/wiki/Open_Virtualization_Format>`__ files
*should* support vdi files, but since they have no identifier URL not
even VirtualBox itself can import them). VirtualBox Guest Additions can
be installed automatically if the ISO is `provided in the
manifest <http://bootstrap-vz.readthedocs.org/en/master/manifest.html#bootstrapper>`__.

177
docs/Makefile
View file

@ -1,177 +0,0 @@
# Makefile for Sphinx documentation
#
# You can set these variables from the command line.
SPHINXOPTS =
SPHINXBUILD = sphinx-build
PAPER =
BUILDDIR = _build
# User-friendly check for sphinx-build
ifeq ($(shell which $(SPHINXBUILD) >/dev/null 2>&1; echo $$?), 1)
$(error The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed, then set the SPHINXBUILD environment variable to point to the full path of the '$(SPHINXBUILD)' executable. Alternatively you can add the directory with the executable to your PATH. If you don't have Sphinx installed, grab it from http://sphinx-doc.org/)
endif
# Internal variables.
PAPEROPT_a4 = -D latex_paper_size=a4
PAPEROPT_letter = -D latex_paper_size=letter
ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
# the i18n builder cannot share the environment and doctrees with the others
I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
.PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest gettext
help:
@echo "Please use \`make <target>' where <target> is one of"
@echo " html to make standalone HTML files"
@echo " dirhtml to make HTML files named index.html in directories"
@echo " singlehtml to make a single large HTML file"
@echo " pickle to make pickle files"
@echo " json to make JSON files"
@echo " htmlhelp to make HTML files and a HTML help project"
@echo " qthelp to make HTML files and a qthelp project"
@echo " devhelp to make HTML files and a Devhelp project"
@echo " epub to make an epub"
@echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter"
@echo " latexpdf to make LaTeX files and run them through pdflatex"
@echo " latexpdfja to make LaTeX files and run them through platex/dvipdfmx"
@echo " text to make text files"
@echo " man to make manual pages"
@echo " texinfo to make Texinfo files"
@echo " info to make Texinfo files and run them through makeinfo"
@echo " gettext to make PO message catalogs"
@echo " changes to make an overview of all changed/added/deprecated items"
@echo " xml to make Docutils-native XML files"
@echo " pseudoxml to make pseudoxml-XML files for display purposes"
@echo " linkcheck to check all external links for integrity"
@echo " doctest to run all doctests embedded in the documentation (if enabled)"
clean:
rm -rf $(BUILDDIR)/*
html:
$(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html
@echo
@echo "Build finished. The HTML pages are in $(BUILDDIR)/html."
dirhtml:
$(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml
@echo
@echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml."
singlehtml:
$(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml
@echo
@echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml."
pickle:
$(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle
@echo
@echo "Build finished; now you can process the pickle files."
json:
$(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json
@echo
@echo "Build finished; now you can process the JSON files."
htmlhelp:
$(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp
@echo
@echo "Build finished; now you can run HTML Help Workshop with the" \
".hhp project file in $(BUILDDIR)/htmlhelp."
qthelp:
$(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp
@echo
@echo "Build finished; now you can run "qcollectiongenerator" with the" \
".qhcp project file in $(BUILDDIR)/qthelp, like this:"
@echo "# qcollectiongenerator $(BUILDDIR)/qthelp/bootstrap-vz.qhcp"
@echo "To view the help file:"
@echo "# assistant -collectionFile $(BUILDDIR)/qthelp/bootstrap-vz.qhc"
devhelp:
$(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp
@echo
@echo "Build finished."
@echo "To view the help file:"
@echo "# mkdir -p $$HOME/.local/share/devhelp/bootstrap-vz"
@echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/bootstrap-vz"
@echo "# devhelp"
epub:
$(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub
@echo
@echo "Build finished. The epub file is in $(BUILDDIR)/epub."
latex:
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
@echo
@echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex."
@echo "Run \`make' in that directory to run these through (pdf)latex" \
"(use \`make latexpdf' here to do that automatically)."
latexpdf:
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
@echo "Running LaTeX files through pdflatex..."
$(MAKE) -C $(BUILDDIR)/latex all-pdf
@echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex."
latexpdfja:
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
@echo "Running LaTeX files through platex and dvipdfmx..."
$(MAKE) -C $(BUILDDIR)/latex all-pdf-ja
@echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex."
text:
$(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text
@echo
@echo "Build finished. The text files are in $(BUILDDIR)/text."
man:
$(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man
@echo
@echo "Build finished. The manual pages are in $(BUILDDIR)/man."
texinfo:
$(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo
@echo
@echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo."
@echo "Run \`make' in that directory to run these through makeinfo" \
"(use \`make info' here to do that automatically)."
info:
$(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo
@echo "Running Texinfo files through makeinfo..."
make -C $(BUILDDIR)/texinfo info
@echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo."
gettext:
$(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale
@echo
@echo "Build finished. The message catalogs are in $(BUILDDIR)/locale."
changes:
$(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes
@echo
@echo "The overview file is in $(BUILDDIR)/changes."
linkcheck:
$(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck
@echo
@echo "Link check complete; look for any errors in the above output " \
"or in $(BUILDDIR)/linkcheck/output.txt."
doctest:
$(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest
@echo "Testing of doctests in the sources finished, look at the " \
"results in $(BUILDDIR)/doctest/output.txt."
xml:
$(SPHINXBUILD) -b xml $(ALLSPHINXOPTS) $(BUILDDIR)/xml
@echo
@echo "Build finished. The XML files are in $(BUILDDIR)/xml."
pseudoxml:
$(SPHINXBUILD) -b pseudoxml $(ALLSPHINXOPTS) $(BUILDDIR)/pseudoxml
@echo
@echo "Build finished. The pseudo-XML files are in $(BUILDDIR)/pseudoxml."

0
docs/__init__.py Normal file
View file

1
docs/_static/.gitignore vendored Normal file
View file

@ -0,0 +1 @@
graph.json

1
docs/_static/graph.json vendored
View file

File diff suppressed because one or more lines are too long

22
docs/_static/taskoverview.coffee vendored
View file

@ -1,10 +1,10 @@
class window.TaskOverview
viewBoxHeight = 800
viewBoxWidth = 200
viewBoxWidth = 800
margins =
top: 100
top: 200
left: 50
bottom: 100
bottom: 200
right: 50
gravity =
lateral: .1
@ -26,7 +26,7 @@ class window.TaskOverview
constructor: ({@selector}) ->
@svg = d3.select(@selector)
.attr('viewBox', "0 0 #{viewBoxWidth} #{viewBoxHeight}")
d3.json '_static/graph.json', @buildGraph
d3.json '../_static/graph.json', @buildGraph
buildGraph: (error, @data) =>
@createDefinitions()
@ -118,15 +118,23 @@ class window.TaskOverview
.selectAll('line').data(layout.links()).enter()
.append('line').attr('marker-end', 'url(#right-arrowhead)')
mouseOver = (d) ->
labels.classed 'hover', (l) -> d is l
nodes.classed 'highlight', (n) -> d.module is n.module
mouseOut = (d) ->
labels.classed 'hover', no
nodes.classed 'highlight', no
nodes = @svg.append('g').attr('class', 'nodes')
.selectAll('g.partition').data(groups).enter()
.append('g').attr('class', 'partition')
.selectAll('circle').data((d) -> d.values).enter()
.append('circle').attr('r', (d) -> d.radius)
.style('fill', (d, i) -> nodeColors(d[nodeColorKey]))
.style('fill', (d) -> nodeColors(d[nodeColorKey]))
.call(layout.drag)
.on('mouseover', (d) -> (labels.filter (l) -> d is l).classed 'hover', true)
.on('mouseout', (d) -> (labels.filter (l) -> d is l).classed 'hover', false)
.on('mouseover', mouseOver)
.on('mouseout', mouseOut)
labels = @svg.append('g').attr('class', 'node-labels')
.selectAll('g.partition').data(groups).enter()

5
docs/_static/taskoverview.less vendored
View file

@ -6,6 +6,11 @@
}
g.nodes circle {
stroke: #000000;
&.highlight {
stroke: #555599;
stroke-width: 2.5px;
fill: #EEAAAA !important;
}
opacity: .9;
stroke-width: 1.5px;
}

1
docs/changelog.rst Normal file
View file

@ -0,0 +1 @@
.. include:: ../CHANGELOG.rst

86
docs/conf.py
View file

@ -30,7 +30,8 @@ sys.path.insert(0, os.path.abspath(os.pardir))
# ones.
extensions = ['sphinx.ext.coverage',
'sphinx.ext.autodoc',
'sphinx.ext.viewcode',
'sphinx.ext.linkcode',
'docs.replace_rtd_links',
]
# Add any paths that contain templates here, relative to this directory.
@ -261,3 +262,86 @@ texinfo_documents = [('index', 'bootstrap-vz', u'bootstrap-vz Documentation',
# If true, do not generate a @detailmenu in the "Top" node's menu.
#texinfo_no_detailmenu = False
# -- Link to rst files scattered throughout the project -------------------
import glob
import os.path
for readme_path in glob.glob('../bootstrapvz/providers/*/README.rst'):
provider_name = os.path.basename(os.path.dirname(readme_path))
include_path = os.path.join('providers', provider_name + '.rst')
path_to_readme = os.path.join('../../bootstrapvz/providers', provider_name, 'README.rst')
with open(include_path, 'w') as include:
include.write('.. include:: ' + path_to_readme)
for readme_path in glob.glob('../bootstrapvz/plugins/*/README.rst'):
plugin_name = os.path.basename(os.path.dirname(readme_path))
include_path = os.path.join('plugins', plugin_name + '.rst')
path_to_readme = os.path.join('../../bootstrapvz/plugins', plugin_name, 'README.rst')
with open(include_path, 'w') as include:
include.write('.. include:: ' + path_to_readme)
# -- Create task overview graph data --------------------------------------
from docs import taskoverview
data = taskoverview.generate_graph_data()
taskoverview.write_data(data, '_static/graph.json')
# -- Substitute links for github with relative links in readthedocs -------
if on_rtd:
pass
# Snatched from here:
# https://sourcegraph.com/github.com/Gallopsled/pwntools@master/.PipPackage/pwntools/.def/docs/source/conf/linkcode_resolve/lines
baseurl = 'https://github.com/andsens/bootstrap-vz'
import subprocess
try:
git_head = subprocess.check_output('git describe --tags 2>/dev/null', shell=True)
except subprocess.CalledProcessError:
try:
git_head = subprocess.check_output('git rev-parse HEAD', shell=True).strip()[:10]
except subprocess.CalledProcessError:
pass
def linkcode_resolve(domain, info):
if domain != 'py':
return None
if not info['module']:
return None
filepath = info['module'].replace('.', '/') + '.py'
fmt_args = {'baseurl': baseurl,
'commit': git_head,
'path': filepath}
import importlib
import inspect
import types
module = importlib.import_module(info['module'])
value = module
for part in info['fullname'].split('.'):
value = getattr(value, part, None)
if value is None:
break
valid_types = (types.ModuleType, types.ClassType, types.MethodType,
types.FunctionType, types.TracebackType,
types.FrameType, types.CodeType)
if isinstance(value, valid_types):
try:
lines, first = inspect.getsourcelines(value)
fmt_args['linestart'] = first
fmt_args['lineend'] = first + len(lines) - 1
return '{baseurl}/blob/{commit}/{path}#L{linestart}-L{lineend}'.format(**fmt_args)
except IOError:
pass
return '{baseurl}/blob/{commit}/{path}'.format(**fmt_args)

0
docs/base/fs.rst → docs/developers/api/base/fs.rst
View file

4
docs/base/index.rst → docs/developers/api/base/index.rst
View file

@ -7,9 +7,9 @@ and handles the gather, sorting and running of tasks.
.. toctree::
:maxdepth: 2
:glob:
fs
pkg
*
Bootstrap information
---------------------

2
docs/base/pkg.rst → docs/developers/api/base/pkg.rst
View file

@ -16,7 +16,7 @@ Sources list
:private-members:
Preferences list
------------
----------------
.. automodule:: bootstrapvz.base.pkg.preferenceslist
:members:
:private-members:

0
docs/common/fs.rst → docs/developers/api/common/fs.rst
View file

0
docs/common/index.rst → docs/developers/api/common/index.rst
View file

0
docs/common/tasks/index.rst → docs/developers/api/common/tasks/index.rst
View file

9
docs/developers/api/index.rst Normal file
View file

@ -0,0 +1,9 @@
API
===
.. toctree::
:maxdepth: 2
base/index
common/index

1
docs/developers/contributing.rst Normal file
View file

@ -0,0 +1 @@
.. include:: ../../CONTRIBUTING.rst

27
docs/howitworks.rst → docs/developers/index.rst
View file

@ -1,6 +1,17 @@
Developers
============
.. toctree::
:maxdepth: 1
contributing
switches
api/index
taskoverview
How bootstrap-vz works
======================
----------------------
Tasks
~~~~~
@ -15,14 +26,14 @@ via attributes. Here is an example:
::
class MapPartitions(Task):
description = 'Mapping volume partitions'
phase = phases.volume_preparation
predecessors = [PartitionVolume]
successors = [filesystem.Format]
description = 'Mapping volume partitions'
phase = phases.volume_preparation
predecessors = [PartitionVolume]
successors = [filesystem.Format]
@classmethod
def run(cls, info):
info.volume.partition_map.map(info.volume)
@classmethod
def run(cls, info):
info.volume.partition_map.map(info.volume)
In this case the attributes define that the task at hand should run
after the ``PartitionVolume`` task — i.e. after volume has been

0
docs/switches.rst → docs/developers/switches.rst
View file

4
docs/taskoverview.rst → docs/developers/taskoverview.rst
View file

@ -5,11 +5,11 @@ Taskoverview
.. raw:: html
<svg id="taskoverview-graph"></svg>
<link rel="stylesheet/less" type="text/css" href="_static/taskoverview.less">
<link rel="stylesheet/less" type="text/css" href="../_static/taskoverview.less">
<script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/less.js/1.5.0/less.min.js"></script>
<script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/coffee-script/1.6.3/coffee-script.min.js"></script>
<script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/d3/3.3.11/d3.js"></script>
<script type="text/coffeescript" src="_static/taskoverview.coffee"></script>
<script type="text/coffeescript" src="../_static/taskoverview.coffee"></script>
<script type="text/coffeescript">
overview = new window.TaskOverview selector: '#taskoverview-graph'
</script>

33
docs/index.rst
View file

@ -1,30 +1,13 @@
.. bootstrap-vz documentation master file, created by
sphinx-quickstart on Sun Mar 23 16:17:28 2014.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
Welcome to bootstrap-vz's documentation!
========================================
Contents:
bootstrap-vz
============
.. toctree::
:maxdepth: 2
:maxdepth: 1
base/index
common/index
plugins/index
intro
manifest
providers/index
guidelines
taskoverview
howitworks
switches
plugins/index
logging
Indices and tables
==================
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`
changelog
developers/index

1
docs/intro.rst Normal file
View file

@ -0,0 +1 @@
.. include:: ../README.rst

1
docs/manifest.rst Normal file
View file

@ -0,0 +1 @@
.. include:: ../manifests/README.rst

3
docs/plugins/.gitignore vendored Normal file
View file

@ -0,0 +1,3 @@
*
!index.rst
!.gitignore

16
docs/plugins/index.rst
View file

@ -1,3 +1,19 @@
Plugins
=======
Plugins are a key feature of bootstrap-vz. Despite their small size
(most plugins do not exceed 100 source lines of code) they can modify
the behavior of bootstrapped systems to a great extent.
Below you will find documentation for all plugins available for
bootstrap-vz. If you cannot find what you are looking for, consider
`developing it yourself <http://bootstrap-vz.readthedocs.org>`__ and
contribute to this list!
.. toctree::
:maxdepth: 1
:glob:
*

3
docs/providers/.gitignore vendored Normal file
View file

@ -0,0 +1,3 @@
*
!index.rst
!.gitignore

17
docs/providers/index.rst
View file

@ -1,3 +1,18 @@
Providers
=========
Plugins are a key feature of bootstrap-vz. Despite their small size
(most plugins do not exceed 100 source lines of code) they can modify
the behavior of bootstrapped systems to a great extent.
Below you will find documentation for all plugins available for
bootstrap-vz. If you cannot find what you are looking for, consider
`developing it yourself <http://bootstrap-vz.readthedocs.org>`__ and
contribute to this list!
.. toctree::
:maxdepth: 1
:glob:
*

19
docs/replace_rtd_links.py Normal file
View file

@ -0,0 +1,19 @@
def setup(app):
app.connect('doctree-resolved', replace_rtd_links)
return {'version': '0.1'}
def replace_rtd_links(app, doctree, fromdocname):
from docutils import nodes
import re
rtd_baseurl = 'http://bootstrap-vz.readthedocs.org/en/master/'
search = re.compile('^' + re.escape(rtd_baseurl) + '(.*)$')
for node in doctree.traverse(nodes.reference):
if 'refuri' not in node:
continue
if not node['refuri'].startswith(rtd_baseurl):
continue
node['refuri'] = re.sub(search, r'\1', node['refuri'])

13
taskoverview.py → docs/taskoverview.py
View file

@ -1,7 +1,11 @@
#!/usr/bin/python
import sys
import os.path
sys.path.append(os.path.join(os.path.dirname(__file__), '..'))
def main(opts):
def generate_graph_data():
from bootstrapvz.base.tasklist import get_all_tasks
tasks = get_all_tasks()
@ -45,13 +49,11 @@ def main(opts):
link[key] = tasks.index(link[key])
return link
data = {'phases': map(mk_phase, phases.order),
return {'phases': map(mk_phase, phases.order),
'modules': map(mk_module, modules),
'nodes': map(mk_node, tasks),
'links': map(mk_link, task_links)}
write_data(data, opts.get('--output', None))
def write_data(data, output_path=None):
import json
@ -73,4 +75,5 @@ if __name__ == '__main__' and __package__ is None:
"""
opts = docopt(usage)
main(opts)
data = generate_graph_data()
write_data(data, opts.get('--output', None))

241
manifests/README.rst Normal file
View file

@ -0,0 +1,241 @@
Manifest
========
| The manifest file is the primary way to interact with bootstrap-vz.
| Every configuration and customization of a Debian installation is
specified in this file.
The manifest format is YAML or JSON. It is near impossible to run the
bootstrapper with an invalid configuration, since every part of the
framework supplies a `json-schema <http://json-schema.org/>`__ that
specifies exactly which configuration settings are valid in different
situations.
Manifest variables
------------------
Many of the settings in the example manifests use strings like
``debian-{system.release}-{system.architecture}-{{"{%y"}}}{{"{%m"}}}{{"{%d"}}}``.
These strings make use of manifest variables, which can cross reference
other settings in the manifest or specific values supplied by the
bootstrapper (e.g. all python date formatting variables are available).
Any reference uses dots to specify a path to the desired manifest
setting. Not all settings support this though, to see whether embedding
a manifest variable in a setting is possible, look for the
*``manifest vars``* label.
Sections
--------
The manifest is split into 7 sections.
Provider
~~~~~~~~
The provider section contains all provider specific settings and the
name of the provider itself.
- **``name``**: target virtualization platform of the installation
*``required``*
Consult the `providers <http://bootstrap-vz.readthedocs.org/en/master/providers>`__ section of the documentation
for a list of valid values.
Bootstrapper
~~~~~~~~~~~~
This section concerns the bootstrapper itself and its behavior. There
are 4 possible settings:
- **``workspace``**: Path to where the bootstrapper should place images
and intermediate files. Any volumes will be mounted under that path.
*``required``*
- **``tarball``**: debootstrap has the option to download all the
software and pack it up in a tarball. When starting the actual
bootstrapping process, debootstrap can then be pointed at that
tarball and use it instead of downloading anything from the internet.
If you plan on running the bootstrapper multiple times, this option
can save you a lot of bandwidth and time. This option just specifies
whether it should create a new tarball or not. It will search for and
use an available tarball if it already exists, regardless of this
setting.
*``optional``*
Valid values: ``true, false``
Default: ``false``
- **``mirror``**: The mirror debootstrap should download software from.
It is advisable to specify a mirror close to your location (or the
location of the host you are bootstrapping on), to decrease latency
and improve bandwidth. If not specified, `the configured aptitude
mirror URL <#packages>`__ is used.
*``optional``*
- **``include_packages``**: Extra packages to be installed during
bootstrap. Accepts a list of package names.
*``optional``*
- **``exclude_packages``**: Packages to exclude during bootstrap phase.
Accepts a list of package names.
*``optional``*
- **``guest_additions``**: This setting is only relevant for the
`virtualbox provider <http://bootstrap-vz.readthedocs.org/en/master/providers/virtualbox.html>`__.
It specifies the path to the VirtualBox Guest Additions ISO, which, when specified,
will be mounted and used to install the VirtualBox Guest Additions.
*``optional``*
Image
~~~~~
The image section configures anything pertaining directly to the image
that will be created.
- **``name``**: The name of the resulting image.
When bootstrapping cloud images, this would be the name visible in
the interface when booting up new instances.
When bootstrapping for VirtualBox or kvm, it's the filename of the
image.
*``required``*
*``manifest vars``*
- **``description``**: Description of the image. Where this setting is
used depends highly on which provider is set. At the moment it is
only used for AWS images.
**``required for ec2 provider``**
*``manifest vars``*
- **``bucket``**: When bootstrapping an S3 backed image for AWS, this
will be the bucket where the image is uploaded to.
**``required for S3 backing``**
- **``region``**: Region in which the AMI should be registered.
**``required for S3 backing``**
System
~~~~~~
This section defines anything that pertains directly to the bootstrapped
system and does not fit under any other section.
- **``architecture``**: The architecture of the system.
Valid values: ``i386, amd64``
*``required``*
- **``bootloader``**: The bootloader for the system. Depending on the
bootmethod of the virtualization platform, the options may be
restricted.
Valid values: ``grub, extlinux, pv-grub``
*``required``*
- **``charmap``**: The default charmap of the system.
Valid values: Any valid charmap like ``UTF-8``, ``ISO-8859-*`` or
``GBK``.
*``required``*
- **``hostname``**: hostname to preconfigure the system with.
*``optional``*
- **``locale``**: The default locale of the system.
Valid values: Any locale mentioned in ``/etc/locale.gen``
*``required``*
- **``release``**: Defines which debian release should be bootstrapped.
Valid values: ``squeeze``, ``wheezy``, ``jessie``, ``sid``,
``oldstable``, ``stable``, ``testing``, ``unstable``
*``required``*
- **``timezone``**: Timezone of the system.
Valid values: Any filename from ``/usr/share/zoneinfo``
*``required``*
Packages
~~~~~~~~
The packages section allows you to install custom packages from a
variety of sources.
- **``install``**: A list of strings that specify which packages should
be installed. Valid values: package names optionally followed by a
``/target`` or paths to local ``.deb`` files.
- **``install_standard``**: Defines if the packages of the
``"Standard System Utilities"`` option of the Debian installer,
provided by `tasksel <https://wiki.debian.org/tasksel>`__, should be
installed or not. The problem is that with just ``debootstrap``, the
system ends up with very basic commands. This is not a problem for a
machine that will not be used interactively, but otherwise it is nice
to have at hand tools like ``bash-completion``, ``less``, ``locate``,
etc.
*``optional``*
Valid values: ``true``, ``false``
Default: ``false``
- **``mirror``**: The default aptitude mirror.
*``optional``*
Default: ``http://http.debian.net/debian``
- **``sources``**: A map of additional sources that should be added to
the aptitude sources list. The key becomes the filename in
``/etc/apt/sources.list.d/`` (with ``.list`` appended to it), while
the value is an array with each entry being a line.
*``optional``*
- **``components``**: A list of components that should be added to the
default apt sources. For example ``contrib`` or ``non-free``
*``optional``*
Default: ``['main']``
- **``trusted-keys``**: List of paths to ``.gpg`` keyrings that should
be added to the aptitude keyring of trusted signatures for
repositories.
*``optional``*
- **``preferences``**: Allows you to pin packages through `apt
preferences <https://wiki.debian.org/AptPreferences>`__. The setting
is an object where the key is the preference filename in
``/etc/apt/preferences.d/``. The key ``main`` is special and refers
to the file ``/etc/apt/preferences``, which will be overwritten if
specified.
*``optional``*
The values are objects with three keys:
- **``package``**: The package to pin (wildcards allowed)
- **``pin``**: The release to pin the package to.
- **``pin-priority``**: The priority of this pin.
Volume
~~~~~~
bootstrap-vz allows a wide range of options for configuring the disk
layout of the system. It can create unpartitioned as well as partitioned
volumes using either the gpt or msdos scheme. At most, there are only
three partitions with predefined roles configurable though. They are
boot, root and swap.
- **``backing``**: Specifies the volume backing. This setting is very
provider specific.
Valid values: ``ebs``, ``s3``, ``vmdk``, ``vdi``, ``raw``
*``required``*
- **``partitions``**: A map of the partitions that should be created on
the volume.
- **``type``**: The partitioning scheme to use. When using ``none``,
only root can be specified as a partition.
Valid values: ``none``, ``gpt``, ``msdos``
*``required``*
- **``root``**: Configuration of the root partition. *``required``*
- **``size``**: The size of the partition. Valid values: Any
datasize specification up to TB (e.g. 5KiB, 1MB, 6TB).
*``required``*
- **``filesystem``**: The filesystem of the partition. When choosing
``xfs``, the ``xfsprogs`` package will need to be installed.
Valid values: ``ext2``, ``ext3``, ``ext4``, ``xfs``
*``required``*
- **``format_command``**: Command to format the partition with. This
optional setting overrides the command bootstrap-vz would normally
use to format the partition. The command is specified as a string
array where each option/argument is an item in that array (much
like the `image\_commands <http://bootstrap-vz.readthedocs.org/en/master/plugins/image_commands.html>`__ plugin).
*``optional``* The following variables are available:
- **``{fs}``**: The filesystem of the partition.
- **``{device_path}``**: The device path of the partition.
- **``{size}``**: The size of the partition.
The default command used by boostrap-vz is
``['mkfs.{fs}', '{device_path}']``.
- **``boot``**: Configuration of the boot partition. The three
settings equal those of the root partition.
*``optional``*
- **``swap``**: Configuration of the swap partition. Since the swap
partition has its own filesystem you can only specify the size for
this partition.
*``optional``*
Plugins
~~~~~~~
The plugins section is a map of plugin names to whatever configuration a
plugin requires. Go to the `plugin section <http://bootstrap-vz.readthedocs.org/en/master/plugins>`__
of the documentation, to see the configuration for a specific plugin.

20
tests/integration/manifests_tests.py
View file

@ -1,12 +1,6 @@
import os
from nose.tools import assert_true
from bootstrapvz.base.manifest import Manifest
MANIFEST_DIR = os.path.join(
os.path.dirname(os.path.realpath(__file__)),
'../../manifests'
)
def test_manifest_generator():
"""
@ -15,10 +9,16 @@ def test_manifest_generator():
Loops through the manifests directory and tests that
each file can successfully be loaded and validated.
"""
for fobj in os.listdir(MANIFEST_DIR):
path = os.path.join(os.path.abspath(MANIFEST_DIR), fobj)
yield validate_manifests, path
import os.path
import glob
manifests = os.path.join(
os.path.dirname(os.path.realpath(__file__)),
'../../manifests'
)
for manifest_path in glob.glob(manifests + '/*.yml'):
yield validate_manifests, manifest_path
for manifest_path in glob.glob(manifests + '/*.json'):
yield validate_manifests, manifest_path
def validate_manifests(path):

7
tox.ini
View file

@ -14,3 +14,10 @@ deps =
nose
nose-cov
commands = nosetests -v tests/integration --with-coverage --cover-package=bootstrapvz --cover-inclusive
[testenv:docs]
changedir = docs
deps =
sphinx
commands =
sphinx-build -W -b html -d _build/html/doctrees . _build/html