Commit graph

265 commits

Author SHA1 Message Date
Anders Ingemann
ece717a79f Fix bugs when picking the build server 2015-04-16 22:15:19 +02:00
Anders Ingemann
cee05e3fd0 Refactor... 2015-04-16 22:15:19 +02:00
Anders Ingemann
3d38ce04ef Move build_servers module into bootstrapvz 2015-04-16 22:15:19 +02:00
Anders Ingemann
909e5cee46 Put remote module into pkg root 2015-04-16 22:15:19 +02:00
Anders Ingemann
209651ef38 Dry run working in tests 2015-04-16 22:15:19 +02:00
Anders Ingemann
a7a3161d66 Require Pyro4 >= 4.30 2015-04-16 22:15:19 +02:00
Anders Ingemann
c5bc45218e Fix documentation 2015-04-16 22:15:19 +02:00
Anders Ingemann
65b9e10ce3 VirtualBox integration 2015-04-16 22:15:19 +02:00
Anders Ingemann
31c453070f Add log source to log messages 2015-04-16 22:15:18 +02:00
Anders Ingemann
0f4c08e51d Refactor, logging, comments 2015-04-16 22:15:18 +02:00
Anders Ingemann
08976ffe07 Remove unused import 2015-04-16 22:15:18 +02:00
Anders Ingemann
1ddc0fbc32 Add comment about random ports 2015-04-16 22:15:18 +02:00
Anders Ingemann
f8d79f06e3 More graceful CallbackServer shutdown 2015-04-16 22:15:18 +02:00
Anders Ingemann
d88d73c7f1 Allow listening on different ports on remote and local 2015-04-16 22:15:18 +02:00
Anders Ingemann
fd07954764 Remove manual deserialization 2015-04-16 22:15:18 +02:00
Anders Ingemann
ed222d579f Little refactor 2015-04-16 22:15:17 +02:00
Anders Ingemann
3542406b91 Fix (de-)serialization 2015-04-16 22:15:17 +02:00
Anders Ingemann
2b33561b82 Don't kill SSH, ask callback server to stop instead of commtimeout 2015-04-16 22:15:17 +02:00
Anders Ingemann
a35ae91b67 Simplify? Hopefully... 2015-04-16 22:15:17 +02:00
Anders Ingemann
282e22c001 debugging 2015-04-16 22:15:17 +02:00
Anders Ingemann
569e1246a5 Deserialize manifest 2015-04-16 22:15:17 +02:00
Anders Ingemann
922cabe80a Serialize exception info by printing it 2015-04-16 22:15:17 +02:00
Anders Ingemann
150b15bb4f (g|s)etstate for manifest 2015-04-16 22:15:17 +02:00
Anders Ingemann
aa9616f4a6 Stop callback server last 2015-04-16 22:15:17 +02:00
Anders Ingemann
16837b38c2 Close thread on both ends! 2015-04-16 22:15:17 +02:00
Anders Ingemann
e271f3e49a Initial work on integration testing started.
The work consists of three parts:
* Allow for bootstrapping remotely,
  this makes it possible to run the tests
  on e.g. OSX with VirtualBox installed
* Make bootstrapping a fully automated process
  where the manifests can be generated by the tests
  and the tests can call the bootstrapper directly in python
* Create a framework wherein instances can be booted up
  using the bootstrapped images and subsequently tested
2015-04-16 22:15:17 +02:00
Anders Ingemann
65ddee99d7 Fix some links 2015-04-12 15:01:06 +02:00
Anders Ingemann
953987fddf Combine documentation from github wiki with sphinx
This is the first step in the effort of combining all documentation about
bootstrap-vz into sphinx while still being able to access it from github
(github can parse rst as well)
2015-04-11 16:40:44 +02:00
Anders Ingemann
d06fbb4a2d Turn on nazi option for sphinx and fix warnings 2015-04-11 16:40:44 +02:00
Jonh Wendell
a5cd6e077d ec2_launch: Allow to deregister the AMI after launching image
If all you want is to test an image or product and to achieve this
you need to generate several images a day, you will end up with
lots of AMI's and snapshots that have no use in the end of the day.

This commit adds the new boolean manifest option "deregister_ami"
that, if True, deletes the recently created AMI and snapshot. So, the
final result will be only the running instance, nothing else.
2015-04-06 14:23:36 -03:00
Jonh Wendell
50d61c735d ec2_launch: Store the instance object directly in the info dictionary
Instead of storing just its ID.

This gives quick access to the recently created instance, which
allows us to simplify the code that needs to fetch the instance object
every time it was necessary.
2015-04-06 14:21:13 -03:00
Jonh Wendell
b934808cce ec2_launch: Simplify the logic for getting the IP address
Use the function waituntil() available in the integration-test branch
to simplify the code that retrieves the IP of the instance.

A 'TODO' note was also added to remember us to merge this function
once it gets merged.
2015-04-06 11:47:15 -03:00
Jonh Wendell
af68be15c6 New plugin: ec2_launch
It adds the ability to automatically launch an EC2 instance
after AMI registration.

It has no mandatory configuration, only optional ones, like
instance type, security groups, etc. They should be documented in
later documentation patches.
2015-03-30 15:36:33 -03:00
Jonh Wendell
8ee0af1a3b Execute entries with a single string as shell commands
Documentation says about that, but it's not implemented like that.

If the command to be executed is an array with just one string, it's
likely the user passed the entire command inside the string.

Plus, they benefit from shell expansion of wildcards like '*'.
2015-03-27 11:23:58 -03:00
Jonh Wendell
1290694f9a Add the manifest "include-source-type" key for packages object
It controls whether to include the 'deb-src' lines in image's
source.list.

Currently they are always included. This patch changes this
behavior by not including them by default; the user must set
this new config to true in order to include them.

This saves a bit of bandwidth in default installations. Also,
the use of src packages is not so usual in ordinary installations.
2015-03-03 19:39:58 -03:00
Anders Ingemann
d046c8e1f7 Merge pull request #194 from Rory-Finnegan/ansible
Added an Ansible plugin, which runs a playbook on the chroot
2015-02-22 16:56:33 +01:00
Anders Ingemann
31d61e778b Fix some more pep8 errors 2015-02-22 16:49:52 +01:00
Anders Ingemann
f48d392df8 Fix pep8 error 2015-02-22 16:44:58 +01:00
Rory Finnegan
c372757104 Added an Ansible plugin, which runs a playbook on the chroot before before build completion.
NOTE: I'm not doing any validation on the opt_flags param and I don't recommend using for more then adding a -vvvv.
Also, I'm purposely excluding the vault flags (which also pretty commonly used) because you shouldn't be baking
private keys and certs into your images. Instead, just avoid running the vault specific code or use the
opt_flags if absolutely necessary.
2015-02-11 21:20:59 +00:00
Jonh Wendell
ea21dcfcec Don't boot quietly on extlinux (aws)
So that we are able to see the System Log through EC2 console
or command line tools.
2015-02-05 10:45:31 -02:00
Anders Ingemann
c2b37c7329 Merge pull request #192 from wrigri/disk-resize
Disable resize on disks larger than 2TB
2015-01-31 18:26:15 +00:00
Rick Wright
0d494fb49e Disable resize on disks larger than 2TB
Change-Id: I9764fe2a06cf47e8c0daf38df41c288c280bd6f7
2015-01-29 16:22:30 -08:00
Jonh Wendell
8418090018 Recover file permissions after shrink a vmdk image
After vmware-vdiskmanager runs, the image is left with a 600
permission. This commit fixes it by preserving the file permission
before the shrink operation.
2015-01-27 21:12:39 -02:00
Brian Mattern
4bd71a2dbe Blacklist floppy module to speed up boot by several seconds.
Add UpdateInitramfs task which is needed for this to take effect.
Enable both tasks for GCE.

Console output before
=====================
[    1.877142] sd 0:0:1:0: [sda] Attached SCSI disk
[    1.880163] sd 0:0:1:0: Attached scsi generic sg0 type 0
[    2.684132] tsc: Refined TSC clocksource calibration: 2500.000 MHz
[    4.824081] floppy0: no floppy controllers found
[    5.103671] work still pending
Begin: Loading essential drivers ... done.
Begin: Running /scripts/init-premount ... done.
Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done.
Begin: Running /scripts/local-premount ... done.
[    5.313107] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null)
...
[    7.751955] alg: No test for crc32 (crc32-pclmul)
[   10.728078] floppy0: no floppy controllers found
[   11.006680] work still pending
[....] Activating swap... done
[   11.258954] EXT4-fs (sda1): re-mounted. Opts: (null)

Console output after
====================
[    1.829785] sd 0:0:1:0: [sda] Attached SCSI disk
[    1.832806] sd 0:0:1:0: Attached scsi generic sg0 type 0
Begin: Loading essential drivers ... done.
Begin: Running /scripts/init-premount ... done.
Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done.
Begin: Running /scripts/local-premount ... done.
[    1.969862] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null)
...
[    2.878920] alg: No test for crc32 (crc32-pclmul)
[....] Activating swap... done
[    2.986642] EXT4-fs (sda1): re-mounted. Opts: (null)

Delint.

Delint
2015-01-24 13:17:42 -08:00
Anders Ingemann
255f0624b4 Merge pull request #175 from andrewbogott/file_copy_plugin
Add the file_copy plugin
2014-12-15 15:33:32 +06:00
Andrew Bogott
81cb82a335 Fix a slightly broken plugins/puppet/manifest-schema.yml 2014-12-15 09:24:14 +00:00
Andrew Bogott
d34aeab6e0 Add the file_copy plugin
File_copy supports creation of dirs and copying of files
from the host system into the image.

bug 168
2014-12-15 09:20:02 +00:00
Noah Fontes
4093693c2e Add support for enhanced networking on EC2.
This change adds a provider option, enhanced_networking,
which installs the Intel virtual networking driver for
SR-IOV using DKMS. It also modifies the EC2 AMI registration
to include support for SR-IOV.
2014-11-29 13:46:57 -08:00
Dan Merino
be5da54b46 Use DOCKER_LOCKEDMEMORY to set ulimit -l before starting Docker. Very important when applications inside the containers need to lock to large amounts of memory. 2014-11-27 16:28:56 -06:00
Dan Lorenc
718ef7ed26 Fix log_check_call in docker_daemon. 2014-11-24 11:24:13 -08:00