Commit graph

158 commits

Author SHA1 Message Date
Anders Ingemann
2b6fefd789 Fix serialization of exceptions 2015-04-16 22:15:20 +02:00
Jonh Wendell
1290694f9a Add the manifest "include-source-type" key for packages object
It controls whether to include the 'deb-src' lines in image's
source.list.

Currently they are always included. This patch changes this
behavior by not including them by default; the user must set
this new config to true in order to include them.

This saves a bit of bandwidth in default installations. Also,
the use of src packages is not so usual in ordinary installations.
2015-03-03 19:39:58 -03:00
Anders Ingemann
31d61e778b Fix some more pep8 errors 2015-02-22 16:49:52 +01:00
Jonh Wendell
ea21dcfcec Don't boot quietly on extlinux (aws)
So that we are able to see the System Log through EC2 console
or command line tools.
2015-02-05 10:45:31 -02:00
Brian Mattern
4bd71a2dbe Blacklist floppy module to speed up boot by several seconds.
Add UpdateInitramfs task which is needed for this to take effect.
Enable both tasks for GCE.

Console output before
=====================
[    1.877142] sd 0:0:1:0: [sda] Attached SCSI disk
[    1.880163] sd 0:0:1:0: Attached scsi generic sg0 type 0
[    2.684132] tsc: Refined TSC clocksource calibration: 2500.000 MHz
[    4.824081] floppy0: no floppy controllers found
[    5.103671] work still pending
Begin: Loading essential drivers ... done.
Begin: Running /scripts/init-premount ... done.
Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done.
Begin: Running /scripts/local-premount ... done.
[    5.313107] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null)
...
[    7.751955] alg: No test for crc32 (crc32-pclmul)
[   10.728078] floppy0: no floppy controllers found
[   11.006680] work still pending
[....] Activating swap... done
[   11.258954] EXT4-fs (sda1): re-mounted. Opts: (null)

Console output after
====================
[    1.829785] sd 0:0:1:0: [sda] Attached SCSI disk
[    1.832806] sd 0:0:1:0: Attached scsi generic sg0 type 0
Begin: Loading essential drivers ... done.
Begin: Running /scripts/init-premount ... done.
Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done.
Begin: Running /scripts/local-premount ... done.
[    1.969862] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null)
...
[    2.878920] alg: No test for crc32 (crc32-pclmul)
[....] Activating swap... done
[    2.986642] EXT4-fs (sda1): re-mounted. Opts: (null)

Delint.

Delint
2015-01-24 13:17:42 -08:00
Noah Fontes
4093693c2e Add support for enhanced networking on EC2.
This change adds a provider option, enhanced_networking,
which installs the Intel virtual networking driver for
SR-IOV using DKMS. It also modifies the EC2 AMI registration
to include support for SR-IOV.
2014-11-29 13:46:57 -08:00
jbergler
57909eb9bc Allow mixed kernel/userspace architecture.
This is to allow building an image for a 64bit machine but with 32bit userspace.

Probably not a common usecase but ideal if you need to address more higher quantities of memory but cant migrate to a full 64bit userspace due to something like ruby eating twice as much memory.
2014-11-20 17:23:28 +00:00
Tim Smith
3410118250 Fix typo in AdjustExpandRootScript
This commit fixes a typo in common.tasks.initd.AdjustExpandRootScript (a
missing double-quote in the search regex) that rendered it ineffective.
2014-10-09 21:05:48 -07:00
Rick Wright
96a1683c26 Fix task ordering to better support customizations
This ensures that preferences are written before apt update and also ensures
that in GCE the backports are added to the sources before the provider-specific
SetPackageRepositories happens.

Change-Id: I3c85f922c49c2a6fbd3c0f2bad1072eff0d098c8
2014-09-08 11:45:55 -07:00
Jimmy Kaplowitz
c412c4cdcf Fix list of tasks and their ordering
- GCE provider wasn't including the GCE SetHostname task, without which
  https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=604883 was
  preventing the hostname from getting set after reboot.
- During the GCE build, one of the GCE cleaning tasks was trying to run
  an apt-get update after the build-time resolv.conf file was removed.
  Fix this ordering by moving the network.Remove* tasks to the
  system_cleaning phase as they should have been all along, and adding
  an appropriate ordering rule for the GCE cleaning task.
- Add the fallback http.debian.net mirror after, not before, our mirror.
- The puppet plugin's ApplyPuppetManifest task specified that it should
  run before the network.Remove* tasks within the system_modification
  phase. Now that those tasks have been moved to a later phase
  (system_cleaning), remove this dependency. I have no puppet manifest
  to test this change, but am including it in hopes of avoiding a
  breakage there. Hopefully someone who uses puppet can test this or at
  least confirm that it's correct.

Change-Id: Ieca97f288f456bab119989f4cbc4c3993a755830
2014-07-24 20:20:54 -07:00
Anders Ingemann
9e61ac94d2 Add a word, to make error message about nbd more understable 2014-07-16 22:08:14 +02:00
Anders Ingemann
e41dbdd807 Rename variable in load_yaml to match load_json 2014-07-16 22:08:13 +02:00
Tiago Ilieve
cc95f146b1 Add AddDefaultSources as AddBackports predecessor 2014-07-13 02:57:37 -03:00
Tiago Ilieve
8f43ee1dc7 Move AddBackports to common.tasks.apt
The `AddBackports` task is needed by many different plugins, so is
better if it is available as a common task.

This closes #96.
2014-07-12 14:42:52 -03:00
Anders Ingemann
e4a9cc837a Differentiate installation of grub 1.99 and grub 2 2014-07-09 23:17:05 +02:00
Anders Ingemann
34bb45bb00 Factor release codename fetching out into common.tools
This allows code that runs before the bootstrapinformation object has been created
to also figure out the release codename
2014-07-09 23:17:05 +02:00
Anders Ingemann
376baae583 Take advantage of the YAML multiline notation 2014-07-09 23:17:04 +02:00
Anders Ingemann
03a0746299 Convert every JSON file to YAML
Lines removed: over 500. Readiblity gained: A shitload
Now you can actually get an overview of a manifest on a single screen height.
I am sure that it will also save a lot of hassle when modifying schema in the future.
No more "expected property name" etc. because of an extraneous comma
Comments are of course natively support, so there's no need for this minify_json hokey pokey
2014-07-09 23:17:04 +02:00
Jimmy Kaplowitz
19240dc201 Support executing commands in a specific cwd
Simply plumbed through to Popen(), which already supports this.

Change-Id: If1fdf0a33c96f3aad42407fdc7c9c9f7d4b95c00
2014-06-19 17:16:39 -07:00
Tiago Ilieve
aa1f8cf189 Update volume image path after move it
The `loopback.MoveImage` task wasn't updating `info.volume.image_path`
after moving the volume image file. That made `volume.Delete` to fail
when called, because it would try to remove a file that didn't exist
anymore.
2014-06-07 12:48:45 -03:00
Jan Brauer
7371c1636b Incorporate feedback 2014-05-28 21:19:15 +02:00
Jan Brauer
18a85c157f AptUpdate has to explicitly depend on WritePreferences 2014-05-28 09:55:09 +02:00
Anders Ingemann
7fe72feb2c Add --assume-yes to apt-get autoremove 2014-05-18 22:27:10 +02:00
Anders Ingemann
f50a8e66ce Merge branch 'master' into development
Conflicts:
	.gitignore
	bootstrapvz/common/tasks/apt.py
	bootstrapvz/plugins/cloud_init/__init__.py
	bootstrapvz/plugins/cloud_init/tasks.py
	tox.ini
2014-05-08 23:00:54 +02:00
Anders Ingemann
16b394d135 Fix fetching of packages.components
Add update from master
2014-05-08 22:58:29 +02:00
Dhananjay Balan
058d4d8627 Be pythonic.
Use sets.update method to add pacakge names.
2014-05-05 00:00:48 +05:30
Dhananjay Balan
2982bfc629 Expose debootstrap include/exclude in manifest.
This is useful in many cases in which the next steps of bootstrapping the image depends on the packages, for e.g to use a non-conventional repository transports like https.
2014-05-04 22:12:03 +05:30
Anders Ingemann
f1994fab1e Don't use Queue to gather log messages in log_call
... instead we just log the messages directly
2014-05-04 14:36:43 +02:00
Anders Ingemann
40f208c151 Fix AdjustExpandRootScript
Also, only add it when it is required
2014-05-04 13:41:16 +02:00
Anders Ingemann
f3d633780f Use string concatenation instead of format()
String concatenation can at times be easier to read that format().
One should choose whichever approach is more readable.
2014-05-04 13:41:15 +02:00
Anders Ingemann
858e325fac Fix minor mistakes in log_call 2014-05-03 23:15:10 +02:00
Anders Ingemann
b25e607388 Move "sections" setting into "packages" section.
Rename it to "components", which is the real name for it (see http://manpages.debian.org/cgi-bin/man.cgi?query=sources.list).
2014-05-03 21:57:12 +02:00
Anders Ingemann
00780844c7 Add a test for log_call.
Make log_call a little shorter as well, stream_readline() is now a closure.
2014-05-03 19:41:55 +02:00
Tiago Ilieve
f43f961de3 Add multiple threads to log_call
Conflicts:
	bootstrapvz/common/tools.py
2014-05-03 19:41:07 +02:00
Anders Ingemann
0a2cd86b4b Add get_standard_rollback_tasks()
This simplifies the the providser rollback_tasks fn considerably
2014-05-03 16:13:49 +02:00
Anders Ingemann
6d2a52f7a0 Create security group (part of standard group) 2014-05-03 16:13:46 +02:00
Anders Ingemann
d5efec8885 Add ssh_group, which configures SSH properly 2014-05-03 16:13:07 +02:00
Anders Ingemann
c81045cc6e Simplify adding tasks
There is now a get_standard_groups function
2014-05-03 16:12:35 +02:00
Tiago Ilieve
47997b4b13 PEP8 fixes 2014-04-27 11:05:53 -03:00
Tiago Ilieve
51a3e488e3 PEP8 fixes 2014-04-27 10:14:02 -03:00
Tiago Ilieve
fe6a5047cd Rename task_sets to task_groups
This closes #17
2014-04-26 17:13:41 -03:00
Tiago Ilieve
1d36255a78 bootstrap method searches for an existing tarball
The tarball bootstrap method now uses an existing tarball if it is
found, regardless of its settings on the manifest, although it will
create a new one only if `"tarball"` is `true`.

This closes #24.
2014-04-26 13:49:47 -03:00
Tomasz Rybak
f8dd196e59 Add missing unstable configuration fixing EC2 image building problems. 2014-04-22 20:09:47 +02:00
Rory Finnegan
708c3d589e Fixed previous yaml load commit and added a tox.ini file.
DETAILS:
- Apparently a potential security issue with yaml.load vs yaml.safe_load.
  Like pickling, yaml.load could result in arbitrary code being run.
- tox.ini file includes the flake8 settings for this project, build spec for 2.7
  and line for eventually testing the project.
2014-04-15 20:29:01 -05:00
Tiago Ilieve
18fce45d91 Move hostname setting from vagrant into common
This closes #19.
2014-04-13 21:18:02 -03:00
Anders Ingemann
69a76f2597 Fix PEP8 errors 2014-04-10 00:27:37 +02:00
Anders Ingemann
a253e643b7 Fix some task descriptions 2014-04-08 23:12:08 +02:00
Anders Ingemann
71b28af3bd Merge branch 'master' into development 2014-04-08 20:03:26 +02:00
Rory Finnegan
60571f66ad Fixed formatting standards on files I've changed.
Used command `find . -name '*.py' | grep -v minify_json | xargs pep8 --ignore=E101,E221,E241,E501,W191`
2014-04-08 11:56:27 -05:00
Rory Finnegan
4b5c2d8c5f Added YAML config support.
- Manifest format parsing is now checked by the file extension ie: .json, .yml or .yaml.
- load_yaml in common/tools is the same as the json version.
- schema checking of manifest still passes (and fails appropriately) like the json manifests.
- I've also included a sample yaml config based off of the debian test json manifest.
2014-04-08 14:45:12 +00:00