Commit graph

1603 commits

Author SHA1 Message Date
Tomasz Melcer
1de10b2f4f minimize_size: make sure the permissions of bindmounts are preserved 2017-07-02 03:33:37 +02:00
Anders Ingemann
266e9e4ae3 Merge pull request #385 from liori/lone-dpkg
Fix lone minimize_size/dpkg without apt
2017-06-25 08:39:39 +02:00
Tomasz Melcer
5d43b9a0e7 Fix lone minimize_size/dpkg without apt 2017-06-25 01:39:17 +02:00
NeatNerdPrime
1f1ebcedb8 Puppet module update (#365)
* #347 - Fix for debconf validator pointing to wrong file.

* reported in https://github.com/andsens/bootstrap-vz/issues/347

  flake8: commands succeeded
  congratulations :)

* # development commit

* trying to fix the packages install with a forced apt-update prior to
installing packages. should fix any issues prior to

* # dev commit

* Trying to parse the commands with a chrotted log_check_call

* # dev commit

* "TypeError: sequence item 2: expected string, list found" , trying to
fix this

* # dev commit - changed the way local packages get installed

This commit changes the way local deb packages get installed.

* rationale: a local deb package mostly includes a deb to configure apt,
e.g.: Puppet apt package. Therefore, after a local dep pkg install , apt
should update itself to be able to install packages that come with the
repo's configured from the local deb package. This assumes you install
LOCAL packages (deb files) first (as a rule of thumb) and AFTER the
REMOTE packages (you include other packages by name)

* reverted command plugin tasks code

* # Dev commit - puppet module update

Goal is to install a puppet 4 agent on the bootstrapped image

* limitations: Only for debian Jessie, installs only puppet4

* # Feature improvement - Puppet module

* a new way to install and configure puppet on a debian jessie
* only tested on debian jessie, for now it only installs the PC1 agent
package.
* future endeavors include all mentioned todo's:
* TODO: plugin must be able to install on any debian release
* TODO: plugin must be able to offer choice of distro package or
apt.puppetlabs.com package
* TODO: plugin must be able to select release version package (vanilla
or PC1)
* TODO: plugin must be able to set up a puppet SERVER , puppetDB
optional
* TODO: plugin have proper linting
* TODO: write test cases

* # Derp commit

* added project files in GitIgnore

* # Dev commit

* added release detection and installs appropriate package.

* # derp commit

* removed unnecessary class parameter.

* # derp dev commit

* add predecessor to ensure repo package gets installed BEFORE the
agent.

* # derp fixes commit

* i must use info.manifest.release
* a little bit refactoring

* # derp commit

* fixed correct predecessor

* # feature & derp dev commit

* added feature to install puppet modules with the plugin

* # Version upgrade commit

New puppet plugin module update

* Plugin can select proper debian release package to install
* Plugin can install modules directly from forge.puppetlabs.com
* Updated documentation with working example 
* Limitations TL;DR: agent software only, PC1 package only, Absolute
pathing, wheezy/jessie only, assumes production environment for modules.
* Several ugly derp bugs squashed.

* * PR fix commit

* Modified puppet readme file as recommended
* added small description in changelog
https://github.com/andsens/bootstrap-vz/pull/365#discussion_r101378921
https://github.com/andsens/bootstrap-vz/pull/365#discussion_r101379039

* # PR fix commit

* Cleaned debugging leftovers
https://github.com/andsens/bootstrap-vz/pull/365#discussion_r101379293

* # PR fix commit

* Removed todo's from code, creating issues on the project issue
tracker.
https://github.com/andsens/bootstrap-vz/pull/365#discussion_r101381742

* # PR fix commit

* Moved EnableAgent task to the bottom as requested
https://github.com/andsens/bootstrap-vz/pull/365#discussion_r101382775

* # PR fix commit

* Removed gitignore file, I have absolutely no idea how that got there.

* # PR fix commit

* Fixed several TOX issues, it's all green now

* #MR-365 - dev commit 

* first try at installing agent with sources/key injection
* New tasks in puppet module: AddPuppetlabsPC1SourcesList;
InstallPuppetlabsPC1ReleaseKey
* it does NOT do any checks. it assumes the url is correct.

* #MR-365 - derp commit

* removed some unused code in tasks

* #365 - tox fix commit

* fixed several tox warnings

* #365 - tox fix commit

* missed one line...

* #365 - tox fix commit

* Noticed the nature of the tox warnings of 'undefined name', fixed.

* # 365 - Task order fix commit

* Some tasks had some invalid predecessors defined, removed.
* when running a test manifest, it fails due to 'NO_PUBKEY
7F438280EF8D349F', trying a hacky way to address this

* #365 - Puppet module update - install with source-key combo

* It now properly installs puppet agent package with sources/keys
instead of package
* Tox is green

* Delete .project

* Delete .pydevproject

* #365 - Puppet module update 

General puppet module update.

* Fixed several issues discussed in the pull request.
* Tox is all green

* General puppet module update.

* This update allows for the installation of the puppetlabs.com agent VERSION 4 from the official apt.puppetlabs.com repo
* Allows for the installation of puppet modules. Useful for masterless setups.
* Puppet modules you declare in your bootstrap-vz manifest are installed with the --force flag.

Limitations and gotcha's:

* Only works for Debian Wheezy & Jessie.
* The Puppetlabs.com agents are only released for i386 and amd64 architectures.
* If you require the puppet 3.x agent, you should not use this module but add "puppet-agent" into the packages list
* You need to add your own puppet.conf file to fit your needs, this can be easily included in the assets directive.
* Assets path is absolute.

* General puppet module update.

* This update allows for the installation of the puppetlabs.com agent VERSION 4 from the official apt.puppetlabs.com repo
* Allows for the installation of puppet modules. Useful for masterless setups.
* Puppet modules you declare in your bootstrap-vz manifest are installed with the --force flag.

Limitations and gotcha's:

* Only works for Debian Wheezy & Jessie.
* The Puppetlabs.com agents are only released for i386 and amd64 architectures.
* If you require the puppet 3.x agent, you should not use this module but add "puppet-agent" into the packages list
* You need to add your own puppet.conf file to fit your needs, this can be easily included in the assets directive.
* Assets path is absolute.

* #365 - Changed the way trusted keys are fetched.

* @andsens is right, keyrings are better added by injecting them into
the image, removing the need to install package 'ca-certificates'

* * Added new feature: puppet module version can now be included in the
manifest.
* modified manifest-schema to reflect proper module installation
* Updated documentation.
* Example manifest added to demonstrate changes
* @andsens is right, keyrings are better added by injecting them into
the image, removing the need to install package 'ca-certificates'

* Fix several requests

* Example manifest moved to a more suitable location, documentation
changed as well.
* A bit more efficient programming.
* reverted file_copy to it's proper state
2017-06-25 00:29:00 +02:00
Anders Ingemann
334a16ee50
Fix equality & identity test for new Debian release 2017-06-17 00:57:19 +02:00
Anders Ingemann
1e36cc1338 Merge pull request #384 from zmarano/master
Promote stretch to stable, add buster as testing, jessie to oldstable.
2017-06-17 00:49:16 +02:00
Zach Marano
7066efe9d1 Add buster as testing and promote stretch to stable, jessie to oldstable. Stretch release to stable is June 17. 2017-06-16 12:31:58 -07:00
Anders Ingemann
c23e73c222 Merge pull request #383 from Exy13/qcow
Add qcow2 as a disk backend for KVM
2017-06-16 18:54:44 +02:00
Brendan Harley
cc69173e52 Add qcow2 as a disk backend for KVM
This allows to build an image with qcow2 as a disk backend.

The update implements a new class that inherits from QEMUVolume and just
needs the extention and file format.
2017-06-16 15:54:51 +02:00
Anders Ingemann
3e5c94f0d4 Merge pull request #381 from Exy13/lvm
Add LVM as a disk backend
2017-06-08 21:55:34 +02:00
Brendan Harley
511a44f72e Add LVM as a disk backend
Enables the use of Logical Volumes as disk backends.

It uses an existing volume group and has no support for creating a new one.
It will not override an existing logical volume and fail gracefully.

The lv is created, activated and then mounted as a loop device.
The boostraping process is then launched on the loop device.
Once the process is completed, the lv is unmounted and desactivated.

The created lv will be deleted should the boostraping process fail.

The lv must be activated before use.

A manifest has been included for testing purposes.
2017-06-08 17:05:48 +02:00
Brendan Harley
df3a200df3 Fix unfailing CheckExternalCommands
On Unix, with shell=True, the shell default to /bin/sh.
Using Popen(['type', command], shell=True) is equivalent to calling
Popen(['/bin/sh', '-c', 'type', command]).
In this case 'command' becomes a positional parameter to the shell,
and not an argument to the command 'type'.

The solution is to pass a single string as parameter.

The problem is that with shell=True, we are never safe from a shell injection,
so it is wiser to use a python only solution.

The package distutils is part of the standard distribution, so it doesn't add
extra dependencies.
The method find_executable has the same behaviour as 'which' on bash.
2017-05-31 22:28:04 +02:00
Anders Ingemann
07a49529b8 Merge pull request #378 from Exy13/loops
Fix loopdevice partitions not being unmapped correctly
2017-05-31 21:45:27 +02:00
Anders Ingemann
6bc92fef8c
MANIFEST.in: Ensure all assets are included during packaging
Fixes: #376
2017-05-29 23:29:23 +02:00
Brendan Harley
a901963200 Fix loopdevice partitions not being unmapped correctly 2017-05-14 21:24:28 +02:00
Anders Ingemann
9ac970b715 Merge pull request #375 from zmarano/master
Enable SCSI block multiqueue for GCE Stretch images.
2017-05-02 22:02:12 +02:00
Zach Marano
c6a6a6e8b1 Enable SCSI block multiqueue for GCE Stretch images. 2017-05-02 12:55:51 -07:00
Anders Ingemann
7af9cd9fb8 Merge pull request #374 from liori/documentation-fixes
fix some typos in documentation
2017-04-23 20:11:55 +02:00
Tomasz Melcer
9b15c52ed6 fix some typos in documentation 2017-04-23 16:59:37 +02:00
Anders Ingemann
0fa83961b8 Merge pull request #372 from zmarano/master
Fix an ordering issue with the expand-root plugin.
2017-04-15 21:07:31 +02:00
Zach Marano
70a2f93d41 Fix an ordering issue with the expand-root plugin. 2017-04-15 10:29:23 -07:00
Anders Ingemann
a40c639c3c
Move some package adding tasks into the preparation phase 2017-04-15 10:01:27 +02:00
Zach Marano
96ebfe11fa Fix bug in Stretch builds. sshd_config no longer contains
PermitRootLogin. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=852781
2017-03-28 20:24:41 +02:00
Anders Ingemann
7ccce031e0 Merge pull request #369 from preeto/admin_user_pubkey_fix
Fix subprocess call to verify admin_user public key
2017-03-16 19:49:00 +01:00
Pritpal Sabharwal
87efcc812b Pass arguments for shell command as elements in a list. When shell=False, subprocess interprets the string as the name of an executable (including spaces) and fails to run. 2017-03-16 10:49:30 +00:00
Anders Ingemann
a6200ff629 Merge pull request #367 from grim-luminal/update-ixgbevf-url
EC2: Update the URL to dowload the ixgbevf driver
2017-03-15 22:21:08 +01:00
Christopher Grim
c87aa22069 EC2: Update the URL to dowload the ixgbevf driver
The URL for the ixgbevf driver is currently returning a 404. Updating
to the new mirror URL gets EC2 builds with enhanced networking support
working again.
2017-03-15 16:57:44 -04:00
Anders Ingemann
dd67184466 Merge pull request #366 from zmarano/master
GCE: Add the file package which many tests expect to exist. It seems to have disappeared in some recent upstream dependency change.
2017-03-08 09:41:59 +01:00
Zach Marano
4d979dfb6b Add the file package which many tests expect to exist- it seems to have
disappeared in some recent upstream dependency change.
2017-03-07 09:54:50 -08:00
Anders Ingemann
605febdeea
cloud-init: Clarify which values are possible for metadata sources
Fixes #364
2017-02-13 23:18:57 +01:00
Anders Ingemann
a4e4ad9954 Merge pull request #355 from n3ph/master
Add mountopts and enable definition of additional partitions
2017-02-13 23:16:08 +01:00
Michael Gerlach
4eb90a544e type is never going to be a partition 2017-02-04 16:15:24 +01:00
n3ph
5cbd083392 Merge pull request #3 from stefan-walluhn/master
fix pep8
2017-02-04 16:08:42 +01:00
Anders Ingemann
5bc4937704
Update docs with boto3 req.
Fixes #362
2017-02-03 15:41:47 +01:00
Stefan
64e1020e6d fix pep8 2017-02-01 20:16:23 +01:00
Michael Gerlach
5a4719c236 remove semicolons.. :-/ 2017-02-01 18:38:37 +01:00
Michael Gerlach
029fc221a1 add additional partitions to docs 2017-01-31 21:54:10 +01:00
Michael Gerlach
c088e1afac cleanup partitioning in schema 2017-01-31 21:45:59 +01:00
Michael Gerlach
ff4918f1e0 sort out UnformattedPartitions nicer 2017-01-31 21:45:40 +01:00
Anders Ingemann
7de914728e Merge pull request #360 from zmarano/master
Create a plugin for expand_root to replace the internal common task.
2017-01-31 20:21:09 +01:00
Zach Marano
74a58c6829 Create a plugin for expand_root to replace the internal common task.
Switch GCE to use this plugin.
2017-01-26 10:18:04 -08:00
Anders Ingemann
29b61ca05a
Bump version to '0.9.11' 2017-01-23 23:47:29 +01:00
Anders Ingemann
6e4722c011
Remove '*****' from InstallENANetworking task desc. 2017-01-23 23:14:39 +01:00
Anders Ingemann
6866adf734 Merge pull request #353 from vorlock/master
ENA support & migrate to boto3
2017-01-23 09:40:04 +01:00
Marcin Kulisz
673260ec63 EC2 provider: fixing snapshot tagging 2017-01-21 22:44:16 +00:00
Marcin Kulisz
d92123c9dd Small update to Stretch EC2 manifest 2017-01-21 21:14:12 +00:00
Marcin Kulisz
6507ccbc0b Adding boto3 to setup.py and minor lint fixes 2017-01-21 21:13:57 +00:00
Marcin Kulisz
ef698b06ca Final fix for ec2_launch plugin 2017-01-21 20:31:49 +00:00
Marcin Kulisz
ed33a5f067 ec2_launch plugin porting to boto3 +
(print_public_ip not working yet)
2017-01-19 19:00:25 +00:00
Marcin Kulisz
07c66e9030 EC2 provider: added ENA driver (installing with +
enhanced_networking set to simple)
2017-01-19 18:53:26 +00:00