From 3ee994d9bf0ffd17e159f4bafe68b9735d7b6ea2 Mon Sep 17 00:00:00 2001 From: Tiago Ilieve Date: Tue, 8 Apr 2014 08:30:43 -0300 Subject: [PATCH 01/10] Add contribution guidelines I've added a _Contributing_ section to the README and a [CONTRIBUTING file](https://github.com/blog/1184-contributing-guidelines). I do not expected for these changes to be merged right away, as they can be seen as a draft from what I've observed in the past weeks. The intention here is not only help the developers that are new to the project, but also make it easier for Anders to deal with new patches. --- CONTRIBUTING.md | 29 +++++++++++++++++++++++++++++ README.md | 7 +++++++ 2 files changed, 36 insertions(+) create mode 100644 CONTRIBUTING.md diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..07f3a5e --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,29 @@ +Contributing +============ + +Do you want to contribute to the bootstrap-vz project? Nice! Here is the basic workflow: + +* Fork this repository. +* Make any changes you want/need. +* Commit your changes. +* Squash the commits if needed. For instance, it is fine if you have multiple commits describing atomic units +of work, but theres no reason to have many commits just because of corrected typos. +* Push to your fork, preferable on a topic branch. + +By this moment, there are two paths that you have to consider: + +If you patch is a new feature, e.g.: plugin, provider, etc. then: + +* Send a pull request to the `development` branch. It will be merged to the `master` branch when we can make +sure that the code is stable. + +If it as a bug/security fix: + +* Send a pull request to the `master` branch. + +Please try to be very descriptive about your changes when you write a pull request, stating what it does, why +it is needed, what use cases do you think it would be useful, etc. + +Be aware that your modifications needs to be properly documented and pushed to the `gh-pages` branch. + +Happy hacking! :-) diff --git a/README.md b/README.md index f6a2a9a..b9c756a 100644 --- a/README.md +++ b/README.md @@ -22,3 +22,10 @@ and learn [how you create a manifest](http://andsens.github.io/bootstrap-vz/mani Developers ---------- The API documentation can be found at [bootstrap-vz.readthedocs.org](http://bootstrap-vz.readthedocs.org). + +Contributing +------------ + +Contribution guidelines are described on the [CONTRIBUTING](CONTRIBUTING.md) file. +There's also a [topic on the documentation](http://andsens.github.io/bootstrap-vz/development.html#coding_style) +regarding the coding style. From 2b22a9ddb37dcb641eafc0d28f913a2d14a19c6c Mon Sep 17 00:00:00 2001 From: Tiago Ilieve Date: Tue, 8 Apr 2014 14:15:28 -0300 Subject: [PATCH 02/10] Add a note about notification on pull requests --- CONTRIBUTING.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 07f3a5e..51f5502 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -22,7 +22,9 @@ If it as a bug/security fix: * Send a pull request to the `master` branch. Please try to be very descriptive about your changes when you write a pull request, stating what it does, why -it is needed, what use cases do you think it would be useful, etc. +it is needed, what use cases do you think it would be useful, etc. Also, be in mind that if you push a new +commit to your pull request, we won't be notified just by this - that will happen only if you make a comment +on the issue. Be aware that your modifications needs to be properly documented and pushed to the `gh-pages` branch. From 7a366e724ce79c8b2225c9f5ffbc403807b6d899 Mon Sep 17 00:00:00 2001 From: Tiago Ilieve Date: Tue, 8 Apr 2014 14:59:32 -0300 Subject: [PATCH 03/10] Add a note about rebasing pull requests --- CONTRIBUTING.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 51f5502..bb06776 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -7,7 +7,7 @@ Do you want to contribute to the bootstrap-vz project? Nice! Here is the basic w * Make any changes you want/need. * Commit your changes. * Squash the commits if needed. For instance, it is fine if you have multiple commits describing atomic units -of work, but theres no reason to have many commits just because of corrected typos. +of work, but there's no reason to have many little commits just because of corrected typos. * Push to your fork, preferable on a topic branch. By this moment, there are two paths that you have to consider: @@ -22,9 +22,10 @@ If it as a bug/security fix: * Send a pull request to the `master` branch. Please try to be very descriptive about your changes when you write a pull request, stating what it does, why -it is needed, what use cases do you think it would be useful, etc. Also, be in mind that if you push a new -commit to your pull request, we won't be notified just by this - that will happen only if you make a comment -on the issue. +it is needed, what use cases do you think it would be useful, etc. You could as well be asked to rebase your +work on the current branch state, so it can be merged cleanly. Also, be in mind that if you push a new commit +to your pull request, we won't be notified just by this - that will happen only if you make a comment on the +issue. Be aware that your modifications needs to be properly documented and pushed to the `gh-pages` branch. From af379c59fd64af176483774a17532935f275886b Mon Sep 17 00:00:00 2001 From: Tiago Ilieve Date: Sun, 13 Apr 2014 22:36:30 -0300 Subject: [PATCH 04/10] Fix some typos on the CONTRIBUTING file --- CONTRIBUTING.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index bb06776..3f62d05 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -14,18 +14,18 @@ By this moment, there are two paths that you have to consider: If you patch is a new feature, e.g.: plugin, provider, etc. then: -* Send a pull request to the `development` branch. It will be merged to the `master` branch when we can make +* Send a pull request to the `development` branch. It will be merged into the `master` branch when we can make sure that the code is stable. -If it as a bug/security fix: +If it is a bug/security fix: * Send a pull request to the `master` branch. Please try to be very descriptive about your changes when you write a pull request, stating what it does, why it is needed, what use cases do you think it would be useful, etc. You could as well be asked to rebase your work on the current branch state, so it can be merged cleanly. Also, be in mind that if you push a new commit -to your pull request, we won't be notified just by this - that will happen only if you make a comment on the -issue. +to your pull request, we won't be notified just by this - that will happen only if you make a new comment on +the issue. Be aware that your modifications needs to be properly documented and pushed to the `gh-pages` branch. From 1781eef692efd730f8b056502cf307b4cb1e5be9 Mon Sep 17 00:00:00 2001 From: Tiago Ilieve Date: Mon, 21 Apr 2014 17:11:41 -0300 Subject: [PATCH 05/10] Add info about gh-pages-dev on CONTRIBUTING --- CONTRIBUTING.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 3f62d05..1894ea4 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -27,6 +27,7 @@ work on the current branch state, so it can be merged cleanly. Also, be in mind to your pull request, we won't be notified just by this - that will happen only if you make a new comment on the issue. -Be aware that your modifications needs to be properly documented and pushed to the `gh-pages` branch. +Be aware that your modifications needs to be properly documented and pushed to the `gh-pages` branch, if they +concern anything done on `master`. Otherwise, they should be sent to the `gh-pages-dev`. Happy hacking! :-) From f8dd196e594daf8f8de61b87b69cf3f1cf8ff7d9 Mon Sep 17 00:00:00 2001 From: Tomasz Rybak Date: Tue, 22 Apr 2014 18:21:46 +0200 Subject: [PATCH 06/10] Add missing unstable configuration fixing EC2 image building problems. --- bootstrapvz/common/tasks/apt.py | 6 +++--- bootstrapvz/common/tasks/network-configuration.json | 2 ++ bootstrapvz/providers/ec2/tasks/packages-kernels.json | 3 +++ 3 files changed, 8 insertions(+), 3 deletions(-) diff --git a/bootstrapvz/common/tasks/apt.py b/bootstrapvz/common/tasks/apt.py index 3051f55..777f635 100644 --- a/bootstrapvz/common/tasks/apt.py +++ b/bootstrapvz/common/tasks/apt.py @@ -28,9 +28,9 @@ class AddDefaultSources(Task): sections = ' '.join(info.manifest.system['sections']) info.source_lists.add('main', 'deb {apt_mirror} {system.release} '+sections) info.source_lists.add('main', 'deb-src {apt_mirror} {system.release} '+sections) - info.source_lists.add('main', 'deb http://security.debian.org/ {system.release}/updates '+sections) - info.source_lists.add('main', 'deb-src http://security.debian.org/ {system.release}/updates '+sections) - if info.manifest.system['release'] not in {'testing', 'unstable'}: + if info.release_codename != 'sid': + info.source_lists.add('main', 'deb http://security.debian.org/ {system.release}/updates '+sections) + info.source_lists.add('main', 'deb-src http://security.debian.org/ {system.release}/updates '+sections) info.source_lists.add('main', 'deb {apt_mirror} {system.release}-updates ' + sections) info.source_lists.add('main', 'deb-src {apt_mirror} {system.release}-updates ' + sections) diff --git a/bootstrapvz/common/tasks/network-configuration.json b/bootstrapvz/common/tasks/network-configuration.json index 4e5dce7..2937384 100644 --- a/bootstrapvz/common/tasks/network-configuration.json +++ b/bootstrapvz/common/tasks/network-configuration.json @@ -8,5 +8,7 @@ "wheezy": ["auto eth0", "iface eth0 inet dhcp"], "jessie": ["auto eth0", + "iface eth0 inet dhcp"], +"sid": ["auto eth0", "iface eth0 inet dhcp"] } diff --git a/bootstrapvz/providers/ec2/tasks/packages-kernels.json b/bootstrapvz/providers/ec2/tasks/packages-kernels.json index 1e42622..adbbf97 100644 --- a/bootstrapvz/providers/ec2/tasks/packages-kernels.json +++ b/bootstrapvz/providers/ec2/tasks/packages-kernels.json @@ -7,6 +7,9 @@ {"i386": "linux-image-686", "amd64": "linux-image-amd64"}, "jessie": + {"i386": "linux-image-686", + "amd64": "linux-image-amd64"}, +"sid": {"i386": "linux-image-686", "amd64": "linux-image-amd64"} } From 72492a6077c68575f713af1ee4303c4a2ad21af8 Mon Sep 17 00:00:00 2001 From: Tiago Ilieve Date: Sat, 26 Apr 2014 16:48:09 -0300 Subject: [PATCH 07/10] Fix SetMetadataSource predecessors Added GenerateLocale as SetMetadataSource predecessor, to avoid any potential locale warnings. Also fixed a space indentation flaw and a string which was being formatted with a non-existent key-argument. This closes #75. --- bootstrapvz/plugins/cloud_init/tasks.py | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/bootstrapvz/plugins/cloud_init/tasks.py b/bootstrapvz/plugins/cloud_init/tasks.py index dde46d6..007ad86 100644 --- a/bootstrapvz/plugins/cloud_init/tasks.py +++ b/bootstrapvz/plugins/cloud_init/tasks.py @@ -2,6 +2,8 @@ from bootstrapvz.base import Task from bootstrapvz.common import phases from bootstrapvz.common.tools import log_check_call from bootstrapvz.common.tasks import apt +from bootstrapvz.common.tasks import locale +import logging import os.path @@ -12,7 +14,6 @@ class AddBackports(Task): @classmethod def run(cls, info): if info.source_lists.target_exists('{system.release}-backports'): - import logging msg = ('{system.release}-backports target already exists').format(**info.manifest_vars) logging.getLogger(__name__).info(msg) else: @@ -52,9 +53,9 @@ class SetUsername(Task): class SetMetadataSource(Task): description = 'Setting metadata source' - #phase = phases.system_modification phase = phases.package_installation - successors = [apt.AptUpdate] + predecessors = [locale.GenerateLocale] + successors = [apt.AptUpdate] @classmethod def run(cls, info): @@ -64,9 +65,8 @@ class SetMetadataSource(Task): source_mapping = {'ec2': 'Ec2'} sources = source_mapping.get(info.manifest.provider, None) if sources is None: - import logging msg = ('No cloud-init metadata source mapping found for provider `{provider}\', ' - 'skipping selections setting.').format(info.manifest.provider) + 'skipping selections setting.').format(provider=info.manifest.provider) logging.getLogger(__name__).warn(msg) return sources = "cloud-init cloud-init/datasources multiselect " + sources From 2e5d291b9c41efd8ae34e15d69a78eeedd513e8d Mon Sep 17 00:00:00 2001 From: Tiago Ilieve Date: Sun, 27 Apr 2014 09:59:11 -0300 Subject: [PATCH 08/10] Add Travis CI By now, it will only test the coding style rules defined for the project, which are basically a few exceptions to the PEP8. --- .gitignore | 3 +++ .travis.yml | 7 +++++++ tox.ini | 10 ++++++++++ 3 files changed, 20 insertions(+) create mode 100644 .travis.yml create mode 100644 tox.ini diff --git a/.gitignore b/.gitignore index ebaa1c3..1da12cf 100644 --- a/.gitignore +++ b/.gitignore @@ -5,6 +5,9 @@ _site/ # When developing for ec2 `vagrant provision' is quite handy /Vagrantfile /.vagrant +# Building the package /build /dist /bootstrap_vz.egg-info +# Testing +/.tox diff --git a/.travis.yml b/.travis.yml new file mode 100644 index 0000000..1327034 --- /dev/null +++ b/.travis.yml @@ -0,0 +1,7 @@ +language: python +python: + - "2.7" +install: + - "pip install tox" +script: + - "tox" diff --git a/tox.ini b/tox.ini new file mode 100644 index 0000000..6d77394 --- /dev/null +++ b/tox.ini @@ -0,0 +1,10 @@ +[flake8] +ignore = E101,E221,E241,E501,W191 +max-line-length = 110 + +[tox] +envlist = flake8 + +[testenv:flake8] +deps = flake8 +commands = flake8 bootstrapvz/ --exclude=minify_json.py From 51a3e488e3cc4c9fa37520cbd8d2f8900520534e Mon Sep 17 00:00:00 2001 From: Tiago Ilieve Date: Sun, 27 Apr 2014 10:14:02 -0300 Subject: [PATCH 09/10] PEP8 fixes --- bootstrapvz/base/fs/__init__.py | 2 +- bootstrapvz/common/__init__.py | 1 - bootstrapvz/common/tasks/apt.py | 8 ++++---- bootstrapvz/common/tasks/filesystem.py | 8 ++++---- bootstrapvz/common/tools.py | 4 ++-- bootstrapvz/plugins/cloud_init/__init__.py | 4 +--- bootstrapvz/plugins/opennebula/__init__.py | 1 - bootstrapvz/plugins/opennebula/tasks.py | 2 +- bootstrapvz/plugins/puppet/tasks.py | 2 +- 9 files changed, 14 insertions(+), 18 deletions(-) diff --git a/bootstrapvz/base/fs/__init__.py b/bootstrapvz/base/fs/__init__.py index b575b87..ca9b1b7 100644 --- a/bootstrapvz/base/fs/__init__.py +++ b/bootstrapvz/base/fs/__init__.py @@ -30,5 +30,5 @@ def load_volume(data, bootloader): 'vmdk': VirtualMachineDisk, 'ebs': EBSVolume } - # Create the volume with the partition map as an argument + # Create the volume with the partition map as an argument return volume_backings.get(data['backing'])(partition_map) diff --git a/bootstrapvz/common/__init__.py b/bootstrapvz/common/__init__.py index f4fa88c..e69de29 100644 --- a/bootstrapvz/common/__init__.py +++ b/bootstrapvz/common/__init__.py @@ -1 +0,0 @@ -from exceptions import * diff --git a/bootstrapvz/common/tasks/apt.py b/bootstrapvz/common/tasks/apt.py index 777f635..d0b9d8f 100644 --- a/bootstrapvz/common/tasks/apt.py +++ b/bootstrapvz/common/tasks/apt.py @@ -26,11 +26,11 @@ class AddDefaultSources(Task): sections = 'main' if 'sections' in info.manifest.system: sections = ' '.join(info.manifest.system['sections']) - info.source_lists.add('main', 'deb {apt_mirror} {system.release} '+sections) - info.source_lists.add('main', 'deb-src {apt_mirror} {system.release} '+sections) + info.source_lists.add('main', 'deb {apt_mirror} {system.release} ' + sections) + info.source_lists.add('main', 'deb-src {apt_mirror} {system.release} ' + sections) if info.release_codename != 'sid': - info.source_lists.add('main', 'deb http://security.debian.org/ {system.release}/updates '+sections) - info.source_lists.add('main', 'deb-src http://security.debian.org/ {system.release}/updates '+sections) + info.source_lists.add('main', 'deb http://security.debian.org/ {system.release}/updates ' + sections) + info.source_lists.add('main', 'deb-src http://security.debian.org/ {system.release}/updates ' + sections) info.source_lists.add('main', 'deb {apt_mirror} {system.release}-updates ' + sections) info.source_lists.add('main', 'deb-src {apt_mirror} {system.release}-updates ' + sections) diff --git a/bootstrapvz/common/tasks/filesystem.py b/bootstrapvz/common/tasks/filesystem.py index 71845f6..482ff3d 100644 --- a/bootstrapvz/common/tasks/filesystem.py +++ b/bootstrapvz/common/tasks/filesystem.py @@ -144,10 +144,10 @@ class FStab(Task): import os.path p_map = info.volume.partition_map mount_points = [{'path': '/', - 'partition': p_map.root, - 'dump': '1', - 'pass_num': '1', - }] + 'partition': p_map.root, + 'dump': '1', + 'pass_num': '1', + }] if hasattr(p_map, 'boot'): mount_points.append({'path': '/boot', 'partition': p_map.boot, diff --git a/bootstrapvz/common/tools.py b/bootstrapvz/common/tools.py index 8721144..2f6a789 100644 --- a/bootstrapvz/common/tools.py +++ b/bootstrapvz/common/tools.py @@ -70,7 +70,8 @@ def config_get(path, config_path): config = config.get(key) return config -def copy_tree(from_path,to_path): + +def copy_tree(from_path, to_path): from shutil import copy import os for abs_prefix, dirs, files in os.walk(from_path): @@ -86,4 +87,3 @@ def copy_tree(from_path,to_path): for path in files: copy(os.path.join(abs_prefix, path), os.path.join(to_path, prefix, path)) - diff --git a/bootstrapvz/plugins/cloud_init/__init__.py b/bootstrapvz/plugins/cloud_init/__init__.py index 3dd8682..e0c17c9 100644 --- a/bootstrapvz/plugins/cloud_init/__init__.py +++ b/bootstrapvz/plugins/cloud_init/__init__.py @@ -14,9 +14,7 @@ def resolve_tasks(taskset, manifest): if manifest.system['release'] in ['wheezy', 'stable']: taskset.add(tasks.AddBackports) - taskset.update([ tasks.SetMetadataSource, - tasks.AddCloudInitPackages, - ]) + taskset.update([tasks.SetMetadataSource, tasks.AddCloudInitPackages]) options = manifest.plugins['cloud_init'] if 'username' in options: diff --git a/bootstrapvz/plugins/opennebula/__init__.py b/bootstrapvz/plugins/opennebula/__init__.py index 4164c31..4ac6056 100644 --- a/bootstrapvz/plugins/opennebula/__init__.py +++ b/bootstrapvz/plugins/opennebula/__init__.py @@ -5,4 +5,3 @@ def resolve_tasks(taskset, manifest): if manifest.system['release'] in ['wheezy', 'stable']: taskset.add(tasks.AddBackports) taskset.update([tasks.AddONEContextPackage]) - diff --git a/bootstrapvz/plugins/opennebula/tasks.py b/bootstrapvz/plugins/opennebula/tasks.py index 202c5a2..410bc90 100644 --- a/bootstrapvz/plugins/opennebula/tasks.py +++ b/bootstrapvz/plugins/opennebula/tasks.py @@ -1,7 +1,6 @@ from bootstrapvz.base import Task from bootstrapvz.common.tasks import apt from bootstrapvz.common import phases -import os class AddBackports(Task): @@ -18,6 +17,7 @@ class AddBackports(Task): info.source_lists.add('backports', 'deb {apt_mirror} {system.release}-backports main') info.source_lists.add('backports', 'deb-src {apt_mirror} {system.release}-backports main') + class AddONEContextPackage(Task): description = 'Adding the OpenNebula context package' phase = phases.preparation diff --git a/bootstrapvz/plugins/puppet/tasks.py b/bootstrapvz/plugins/puppet/tasks.py index dca4cd7..2627e69 100644 --- a/bootstrapvz/plugins/puppet/tasks.py +++ b/bootstrapvz/plugins/puppet/tasks.py @@ -2,6 +2,7 @@ from bootstrapvz.base import Task from bootstrapvz.common import phases from bootstrapvz.common.tasks import apt from bootstrapvz.common.tasks import network +from bootstrapvz.common.tools import sed_i import os @@ -82,7 +83,6 @@ class ApplyPuppetManifest(Task): 'puppet', 'apply', manifest_path]) os.remove(manifest_dst) - from bootstrapvz.common.tools import sed_i hosts_path = os.path.join(info.root, 'etc/hosts') sed_i(hosts_path, '127.0.0.1\s*{hostname}\n?'.format(hostname=hostname), '') From 17b0e6e57a16ff1f14b1e01c7c9a34c84d743949 Mon Sep 17 00:00:00 2001 From: Tiago Ilieve Date: Sun, 27 Apr 2014 10:50:07 -0300 Subject: [PATCH 10/10] Add information about coding style checking Added a note about this to the CONTRIBUTING.md file. [skip ci] --- CONTRIBUTING.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 1894ea4..ff303fd 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -5,6 +5,10 @@ Do you want to contribute to the bootstrap-vz project? Nice! Here is the basic w * Fork this repository. * Make any changes you want/need. +* Check the coding style of your changes using [tox](http://tox.readthedocs.org/), running `tox -e flake8` and +fixing any warnings that may appear. This will be done again by +[Travis CI](https://travis-ci.org/andsens/bootstrap-vz) later when you send a pull request, so it's better if +you check this before. * Commit your changes. * Squash the commits if needed. For instance, it is fine if you have multiple commits describing atomic units of work, but there's no reason to have many little commits just because of corrected typos.