Commit graph

74 commits

Author SHA1 Message Date
Zach Marano
84f258e748 Fixes to create stretch images. Add stretch manifest for GCE. 2017-01-10 13:30:31 -08:00
Nicolas Braud-Santoni
90da634226
Systematically use rel_path for relative paths 2016-09-12 19:11:20 +02:00
Zach Marano
1f747788c1 Fix missing source repo in GCE images. Add back baseline apt cache as
well. Add in additional GCE guest package into manifests (init files
were split into their own package to make the python package adhere to
pypi standards).
2016-07-21 14:14:42 -07:00
Anders Ingemann
cf6234bafd Split grub config into small tasks
The way boot options for linux and config params for grub
were configured gave rise to quite a few bugs.
The configuration has now been abstracted so that
options can be added without interfering with the work
of other tasks (no more sed_i!)
2016-06-04 18:48:54 +02:00
Anders Ingemann
f62c8ade99 Convert indentation from tabs to spaces (4)
Up until now I didn't see the point of using spaces for indentation.
However, the previous commit (a18bec3) was quite eye opening.
Given that python is an indentation aware language, the amount of
mistakes that went unnoticed because tabs and spaces were used
at the same time (tabs for indentation and spaces for alignment)
were unacceptable.

E101,W191 have been re-enable in the tox flake8 checker and
the documentation has been modified accordingly.

The following files have been left as-is:
* bootstrapvz/common/assets/extlinux/extlinux.conf
* bootstrapvz/common/assets/init.d/expand-root
* bootstrapvz/common/assets/init.d/generate-ssh-hostkeys
* bootstrapvz/common/assets/init.d/squeeze/generate-ssh-hostkeys
* bootstrapvz/plugins/docker_daemon/assets/init.d/docker
* bootstrapvz/providers/ec2/assets/bin/growpart
* bootstrapvz/providers/ec2/assets/grub.d/40_custom
* bootstrapvz/providers/ec2/assets/init.d/ec2-get-credentials
* bootstrapvz/providers/ec2/assets/init.d/ec2-run-user-data
* docs/_static/taskoverview.coffee
* docs/_static/taskoverview.less
* tests/unit/subprocess.sh
2016-06-04 11:38:16 +02:00
Anders Ingemann
2d6a026160 Fix a slew of indentation & spacing issues 2016-06-04 11:38:16 +02:00
Zach Marano
40ec27ad16 Overhaul GCE image build. 2016-06-02 18:16:57 -07:00
Anders Ingemann
3c1999f809 Merge pull request #316 from justinsb/fix/gce_upload
Fixes for uploading a GCE image
2016-05-23 21:57:47 +02:00
Justin Santa Barbara
3036eb41c1 GCE: Name the image as specified in the manifest
We were hard coding an image name format for GCE, but that doesn't match
what other providers (e.g. AWS) do.

Given that image upload did not work at all previously, no reason to
preserve the existing unusual behaviour.
2016-04-19 11:35:11 -04:00
Justin Santa Barbara
be595efd99 GCE: Rename image in tar file to disk.raw
To register with GCE, the image must be named disk.raw, or else the
gcloud create images command fails.
2016-04-19 11:34:53 -04:00
Justin Santa Barbara
4b34cb2615 GCE: Documentation improvements
Small tweaks
* Add note that gce_project only works with gcs_destination
* Google's storage is called "Google Cloud Storage"
* Minor grammar tweaks
2016-04-19 11:34:53 -04:00
Justin Santa Barbara
34ee0c1e8d GCE: Allow templating of image description
Expand the description template before passing it to gcloud
2016-04-19 11:34:53 -04:00
Justin Santa Barbara
3e3d59ddf1 GCE: Fix upload of image to gcloud
Two small fixes

* syntax is `gcloud compute images`, not `gcloud compute image`
* --source-uri=<uri> should be a single argument
2016-04-19 11:34:46 -04:00
Zach Marano
f1d6cd9d9e Disable SSH root login on GCE images. 2016-04-18 13:31:37 -07:00
Anders Ingemann
83b783976c Merge pull request #308 from pmarks-net/master
Do not remove ::1 from the loopback interface.
2016-04-04 23:00:27 +02:00
Zach Marano
070f678807 Add busybox as a required package for GCE. 2016-03-28 21:53:52 -07:00
Paul Marks
15c7d1ccb3 Do not remove ::1 from the loopback interface.
An environment with AF_INET6 sockets but no loopback interface creates
nothing but pain.

If an IPv4 server binds to 0.0.0.0:8080, clients may connect to
0.0.0.0:8080, which automatically picks 127.0.0.1 as a source address.
However, when a server binds to [::]:8080, the absence of ::1 causes
clients to fail with ENETUNREACH.

For a demonstration, run the following in a python shell:

import socket
s = socket.socket(socket.AF_INET6, socket.SOCK_STREAM, 0)
s.bind(("", 0))
print s.getsockname()  # Example: ('::', 39079, 0, 0)
s.listen(10)
c = socket.socket(socket.AF_INET6, socket.SOCK_STREAM, 0)
c.connect(s.getsockname())
print c.getsockname(), c.getpeername()

This yields the following error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/socket.py", line 224, in meth
    return getattr(self._sock,name)(*args)
socket.error: [Errno 101] Network is unreachable
2016-03-24 17:14:17 -07:00
hitsumabushi
ac7595bdfb fix document toctree 2016-03-15 11:13:05 +09:00
Anders Ingemann
d62e20c763 Merge pull request #303 from zmarano/master
Again fix the expand-root script for GCE.
2016-03-04 01:26:07 +01:00
Anders Ingemann
6bb654b79f Compat with new sphinx
Changed a lot of code:: to code-block::
Also added sphinx_rtd_theme to deps in tox
2016-03-04 01:21:52 +01:00
Zach Marano
d485ef681a Fix typo 2016-03-03 14:29:42 -08:00
Zach Marano
b590a02a8c Fix import. 2016-03-03 14:14:54 -08:00
Zach Marano
2b629d784c Again fix the expand-root script for GCE. The volume is not /dev/loop0
but it should be /dev/sda.
2016-03-03 13:59:01 -08:00
Zach Marano
31a98badcb Fix unused import. 2016-03-02 16:28:11 -08:00
Zach Marano
6c03860eac Fix GCE expand-root since commit/2ece5a66e1e36f30bef3a675e8287ab8c1bcc04f 2016-03-02 12:54:06 -08:00
Zach Marano
c34d7d11c8 Only include bare essential packages in the GCE provider task. Move
extras to the manifest files.
2016-02-29 10:38:34 -08:00
Zach Marano
74a8aae6ac Cleanup package list and add in packages that were being added outside
of bootstrap-vz. Remove i386 kernels that GCE does not support as well
as squeeze, and add in a stretch kernel.
2016-02-26 16:24:20 -08:00
Zach Marano
977b1c23a3 Change location of GCE apt repo's (this is an interim change).
hange gcutil command to gcloud as gcutil is deprecated.
2015-12-15 11:20:49 -08:00
Anders Ingemann
c880a6849a Remove image section in manifest
All provider specific settings have been moved to the
provider section. The image name itself is now located
at the top level and called "name". It is required for all providers.
2015-12-13 19:41:18 +01:00
Anders Ingemann
52c05e5df6 Move docs about image description to gce & ec2 providers 2015-12-13 19:41:18 +01:00
Tiago Ilieve
3d3c303ee1 Changes 'MoveImage' from loopback to image 2015-10-18 16:35:11 -02:00
Tiago Ilieve
10742f8ca3 gce: the right name isn't GCE or GSE, it's GCS[1]
[1]: https://cloud.google.com/storage/
2015-10-18 16:31:12 -02:00
Tiago Ilieve
0068cdc1a8 gce: fix small typo on UploadImage 2015-10-16 00:06:58 -03:00
Rick Wright
1e2fe02847 Fix a problem with Debian 8 on GCE with >2TB disks
This change makes two fixes. First, it fixes a race condition on Debian
8 GCE instances which caused instances with >2TB disks to fail. Second,
it adds the required "PREREQ" header to the script, which was previously
missing and caused errors at build time. These two issues are somewhat
unrelated to each other, but debugging the first led to finding the
second.
2015-09-23 16:55:16 -07:00
Rick Wright
ef429f5e79 Install GCE packages before Manifest packages. 2015-06-22 14:28:22 -07:00
Zach Marano
7484236282 Change the order we clean up the apt state- thereby leaving a current as
of the image build set of apt list files on the resulting image.
2015-06-17 18:12:07 -07:00
Zach Marano
d3a1d22290 Add support for Debian 8 in GCE.
Add support for GPT disks.
Change the default block scheduler to noop for improved performance.
2015-05-19 13:19:12 -07:00
Anders Ingemann
989f33c226 Kill the initalize() function. Nobody uses it except ec2
The boto debug messages are now silenced with a task.
2015-05-03 12:31:44 +02:00
Anders Ingemann
1f6f23e680 Fix #104: Don't verify default target when adding packages 2015-04-29 23:51:55 +02:00
Anders Ingemann
71c7d445ad Fix #217, by introducing class for comparison of releases 2015-04-29 21:38:06 +02:00
Anders Ingemann
5823c9119b Split grub and extlinux installs into separate modules 2015-04-16 22:21:12 +02:00
Anders Ingemann
953987fddf Combine documentation from github wiki with sphinx
This is the first step in the effort of combining all documentation about
bootstrap-vz into sphinx while still being able to access it from github
(github can parse rst as well)
2015-04-11 16:40:44 +02:00
Anders Ingemann
c2b37c7329 Merge pull request #192 from wrigri/disk-resize
Disable resize on disks larger than 2TB
2015-01-31 18:26:15 +00:00
Rick Wright
0d494fb49e Disable resize on disks larger than 2TB
Change-Id: I9764fe2a06cf47e8c0daf38df41c288c280bd6f7
2015-01-29 16:22:30 -08:00
Brian Mattern
4bd71a2dbe Blacklist floppy module to speed up boot by several seconds.
Add UpdateInitramfs task which is needed for this to take effect.
Enable both tasks for GCE.

Console output before
=====================
[    1.877142] sd 0:0:1:0: [sda] Attached SCSI disk
[    1.880163] sd 0:0:1:0: Attached scsi generic sg0 type 0
[    2.684132] tsc: Refined TSC clocksource calibration: 2500.000 MHz
[    4.824081] floppy0: no floppy controllers found
[    5.103671] work still pending
Begin: Loading essential drivers ... done.
Begin: Running /scripts/init-premount ... done.
Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done.
Begin: Running /scripts/local-premount ... done.
[    5.313107] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null)
...
[    7.751955] alg: No test for crc32 (crc32-pclmul)
[   10.728078] floppy0: no floppy controllers found
[   11.006680] work still pending
[....] Activating swap... done
[   11.258954] EXT4-fs (sda1): re-mounted. Opts: (null)

Console output after
====================
[    1.829785] sd 0:0:1:0: [sda] Attached SCSI disk
[    1.832806] sd 0:0:1:0: Attached scsi generic sg0 type 0
Begin: Loading essential drivers ... done.
Begin: Running /scripts/init-premount ... done.
Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done.
Begin: Running /scripts/local-premount ... done.
[    1.969862] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null)
...
[    2.878920] alg: No test for crc32 (crc32-pclmul)
[....] Activating swap... done
[    2.986642] EXT4-fs (sda1): re-mounted. Opts: (null)

Delint.

Delint
2015-01-24 13:17:42 -08:00
Tim Smith
7944db886f Enable auto-rootdisk growth on gce-backports.
This commit adds the cloud-initramfs-growroot package to the
installation list for GCE images with backports enabled, and updates
the gce tasklist to add the /etc/init.d/expand-root script
(with provider-appropriate touch-ups) to the image.
2014-10-13 10:02:15 -07:00
Rick Wright
96a1683c26 Fix task ordering to better support customizations
This ensures that preferences are written before apt update and also ensures
that in GCE the backports are added to the sources before the provider-specific
SetPackageRepositories happens.

Change-Id: I3c85f922c49c2a6fbd3c0f2bad1072eff0d098c8
2014-09-08 11:45:55 -07:00
Jimmy Kaplowitz
89e1a701eb Disable SSH password auth on GCE
This change was unintentional but occurred as part of GCE's transition
from build-debian-cloud to bootstrap-vz.

Might be replaced later with a similar change that applies to all
bootstrap-vz providers, based on the opinion of the debian-cloud list.

Change-Id: I72a694c49f32df06252d9cc01b1d5c7cfc015347
2014-08-14 18:10:54 -07:00
Jimmy Kaplowitz
3bfe9dddf9 Reinstate hostname hook for GCE
This DHCP exit hook to shorten the system hostname on GCE was previously
installed by build-debian-cloud and bootstrap-vz, but seems to have been
inadvertently removed in commit c81045cc6e
as part of a broad cross-cloud cleanup. Again, this was caught by our
validation tests, and we might be done with the fixes at this point.

In this commit, I'm reinstating the hook with a name change and an
explanatory comment, to reduce the risk of this vanishing accidentally
in the future.

Change-Id: I4e7268f8b9ab3b2a7fc8b510898c6fbdd685aa53
2014-08-05 17:56:33 -07:00
Anders Ingemann
9f46bdc519 Merge pull request #131 from jkaplowitz/cloudsdk
Install Cloud SDK, which includes gcutil and gsutil.
2014-08-05 17:12:51 -06:00