Gems/Python Packages Updaten

Erstmal herausfinden was installiert ist:

root@puppet ~ # pip list
backports.ssl-match-hostname (3.4.0.2)
configobj (5.0.6)
decorator (3.4.0)
Flask (0.10.1)
Flask-WTF (0.10.3)
gunicorn (19.1.1)
iniparse (0.4)
IPy (0.81)
itsdangerous (0.24)
javapackages (1.0.0)
Jinja2 (2.7.3)
lxml (3.4.1)
MarkupSafe (0.23)
policycoreutils-default-encoding (0.1)
puppetboard (0.0.4)
pycurl (7.19.5)
pygobject (3.8.2)
pygpgme (0.3)
pyliblzma (0.5.3)
pypuppetdb (0.1.1)
pyudev (0.16.1)
pyxattr (0.5.3)
requests (2.5.0)
seobject (0.1)
sepolicy (1.1)
six (1.8.0)
slip (0.4.0)
slip.dbus (0.4.0)
urlgrabber (3.10)
Werkzeug (0.9.6)
wsgiref (0.1.2)
WTForms (2.0.1)
yum-metadata-parser (1.1.4)
root@puppet ~ #

Dann feststellen was davon nicht mehr aktuell ist:

root@puppet ~ # pip list --outdated
Could not find any downloads that satisfy the requirement yum-metadata-parser
Could not find any downloads that satisfy the requirement policycoreutils-default-encoding
Could not find any downloads that satisfy the requirement sepolicy
Could not find any downloads that satisfy the requirement slip.dbus
Could not find any downloads that satisfy the requirement javapackages
IPy (Current: 0.81 Latest: 0.82)
Could not find any downloads that satisfy the requirement seobject
root@puppet ~ #

Pakete die hier nicht gefunden werden können wurden über die Paketverwaltung (in diesem Fall yum) installiert und nicht über pip/easy_install. Geupdatet werden können die Pakete wie folgt:

root@puppet ~ # pip install --upgrade ipy
Downloading/unpacking ipy from https://pypi.python.org/packages/source/I/IPy/IPy-0.82.tar.gz#md5=0625e884df8bde533b54a88c85cf5029
  Downloading IPy-0.82.tar.gz
  Running setup.py egg_info for package ipy
    
Installing collected packages: ipy
  Found existing installation: IPy 0.81
    Uninstalling IPy:
      Successfully uninstalled IPy
  Running setup.py install for ipy
    
Successfully installed ipy
Cleaning up...
root@puppet ~ #

Aktuell liefert pip leider keine Möglichkeit automatisiert alle Pakete zu updaten. Das ganze nun analog für Ruby, erst die Gems auflisten:

root@puppet ~ # gem list 

*** LOCAL GEMS ***

activemodel (4.1.8)
activesupport (4.1.8)
bigdecimal (1.2.5, 1.2.0)
builder (3.2.2)
bundler (1.7.7)
colored (1.2)
cri (2.6.1, 2.5.0)
ethon (0.7.1)
faraday (0.9.0, 0.8.9)
faraday_middleware (0.9.1)
faraday_middleware-multi_json (0.0.6)
ffi (1.9.6)
git (1.2.8)
her (0.7.2)
i18n (0.6.11)
io-console (0.4.2)
json (1.8.1, 1.7.7)
json_pure (1.8.1)
kgio (2.9.2)
log4r (1.1.10)
minitest (5.4.3)
multi_json (1.10.1, 1.8.4)
multipart-post (2.0.0, 1.2.0)
net-ssh (2.9.1)
psych (2.0.6, 2.0.0)
puppet_forge (1.0.3)
r10k (1.4.0, 1.3.5)
rack (1.5.2)
raindrops (0.13.0)
rake (10.4.1, 0.9.6)
rdoc (4.1.2, 4.0.0)
systemu (2.6.4, 2.5.2)
thread_safe (0.3.4)
typhoeus (0.6.9)
tzinfo (1.2.2)
unicorn (4.8.3)
root@puppet ~ #

Updates finden:

root@puppet ~ # gem outdated
bundler (1.7.7 < 1.7.8)
psych (2.0.6 < 2.0.8)
rake (10.4.1 < 10.4.2)
rdoc (4.1.2 < 4.2.0)
root@puppet ~ #

Ein einzelnes Gem updaten:

root@puppet ~ # gem update psych
Updating installed gems
Updating psych
Fetching: psych-2.0.8.gem (100%)
Building native extensions.  This could take a while...
Successfully installed psych-2.0.8
Parsing documentation for psych-2.0.8
Installing ri documentation for psych-2.0.8
Installing darkfish documentation for psych-2.0.8
Done installing documentation for psych after 3 seconds
Gems updated: psych
root@puppet ~ #

Oder alle Gems:

root@puppet ~ # gem update
Updating installed gems
Updating bundler
Fetching: bundler-1.7.8.gem (100%)
Successfully installed bundler-1.7.8
Parsing documentation for bundler-1.7.8
Installing ri documentation for bundler-1.7.8
Installing darkfish documentation for bundler-1.7.8
Done installing documentation for bundler after 7 seconds
Updating rake
Fetching: rake-10.4.2.gem (100%)
Successfully installed rake-10.4.2
Parsing documentation for rake-10.4.2
Installing ri documentation for rake-10.4.2
Installing darkfish documentation for rake-10.4.2
Done installing documentation for rake after 5 seconds
Updating rdoc
Fetching: rdoc-4.2.0.gem (100%)
Depending on your version of ruby, you may need to install ruby rdoc/ri data:

<= 1.8.6 : unsupported
 = 1.8.7 : gem install rdoc-data; rdoc-data --install
 = 1.9.1 : gem install rdoc-data; rdoc-data --install
>= 1.9.2 : nothing to do! Yay!
Successfully installed rdoc-4.2.0
Parsing documentation for rdoc-4.2.0
Installing ri documentation for rdoc-4.2.0
Installing darkfish documentation for rdoc-4.2.0
Done installing documentation for rdoc after 22 seconds
Gems updated: bundler rake rdoc
root@puppet ~ #

Hier werden keine alten Versionen automatisch deinstalliert, dies kann man allerdings manuell triggern. gem beachtet dabei die Abhängigkeiten einzelner gems zu anderen auf ältere Versionen und deinstalliert diese dann nicht. Um zu sehen welche Gems deinstalliert werden kann man einen dry Run machen:

root@puppet ~ # gem cleanup -d
Cleaning up installed gems...
Dry Run Mode: Would uninstall typhoeus-0.6.9
Dry Run Mode: Would uninstall rdoc-4.0.0
Dry Run Mode: Would uninstall rdoc-4.1.2
Dry Run Mode: Would uninstall rake-10.4.1
Dry Run Mode: Would uninstall rack-1.5.2
Dry Run Mode: Would uninstall r10k-1.4.0
Dry Run Mode: Would uninstall puppet_forge-1.0.3
Dry Run Mode: Would uninstall psych-2.0.0
Dry Run Mode: Would uninstall psych-2.0.6
Dry Run Mode: Would uninstall psych-2.0.8
Dry Run Mode: Would uninstall psych-2.0.9
Dry Run Mode: Would uninstall net-ssh-2.9.1
Dry Run Mode: Would uninstall minitest-5.4.3
Dry Run Mode: Would uninstall kgio-2.9.2
Dry Run Mode: Would uninstall json_pure-1.8.1
Dry Run Mode: Would uninstall json-1.7.7
Dry Run Mode: Would uninstall her-0.7.2
Dry Run Mode: Would uninstall git-1.2.8
Dry Run Mode: Would uninstall git-1.2.9
Dry Run Mode: Would uninstall faraday-0.9.0
Dry Run Mode: Would uninstall ethon-0.7.1
Dry Run Mode: Would uninstall cri-2.5.0
Dry Run Mode: Would uninstall bundler-1.7.7
Dry Run Mode: Would uninstall bundler-1.7.8
Dry Run Mode: Would uninstall bundler-1.7.9
Dry Run Mode: Would uninstall bundler-1.7.11
Dry Run Mode: Would uninstall bigdecimal-1.2.0
Dry Run Mode: Would uninstall minitest-5.5.0
Dry Run Mode: Would uninstall json-1.8.1
Dry Run Mode: Would uninstall i18n-0.6.11
Dry Run Mode: Would uninstall typhoeus-0.6.9
Dry Run Mode: Would uninstall rdoc-4.0.0
Dry Run Mode: Would uninstall rdoc-4.1.2
Dry Run Mode: Would uninstall rake-10.4.1
Dry Run Mode: Would uninstall rack-1.5.2
Dry Run Mode: Would uninstall r10k-1.4.0
Dry Run Mode: Would uninstall puppet_forge-1.0.3
Dry Run Mode: Would uninstall psych-2.0.0
Dry Run Mode: Would uninstall psych-2.0.6
Dry Run Mode: Would uninstall psych-2.0.8
Dry Run Mode: Would uninstall psych-2.0.9
Dry Run Mode: Would uninstall net-ssh-2.9.1
Dry Run Mode: Would uninstall minitest-5.4.3
Dry Run Mode: Would uninstall kgio-2.9.2
Dry Run Mode: Would uninstall json_pure-1.8.1
Dry Run Mode: Would uninstall json-1.7.7
Dry Run Mode: Would uninstall her-0.7.2
Dry Run Mode: Would uninstall git-1.2.8
Dry Run Mode: Would uninstall git-1.2.9
Dry Run Mode: Would uninstall faraday-0.9.0
Dry Run Mode: Would uninstall ethon-0.7.1
Dry Run Mode: Would uninstall cri-2.5.0
Dry Run Mode: Would uninstall bundler-1.7.7
Dry Run Mode: Would uninstall bundler-1.7.8
Dry Run Mode: Would uninstall bundler-1.7.9
Dry Run Mode: Would uninstall bundler-1.7.11
Dry Run Mode: Would uninstall bigdecimal-1.2.0
Dry Run Mode: Would uninstall minitest-5.5.0
Dry Run Mode: Would uninstall json-1.8.1
Dry Run Mode: Would uninstall i18n-0.6.11
Clean Up Complete
root@puppet ~ #

und das eigentliche deinstallieren geht dann ohne -d:

root@puppet ~ # gem cleanup
Cleaning up installed gems...
Attempting to uninstall typhoeus-0.6.9
Successfully uninstalled typhoeus-0.6.9
Attempting to uninstall rdoc-4.0.0
Unable to uninstall rdoc-4.0.0:
	Gem::InstallError: rdoc is not installed in GEM_HOME, try:
	gem uninstall -i /usr/share/gems rdoc
Attempting to uninstall rdoc-4.1.2
Successfully uninstalled rdoc-4.1.2
Attempting to uninstall rake-10.4.1
Successfully uninstalled rake-10.4.1
Attempting to uninstall rack-1.5.2
Successfully uninstalled rack-1.5.2
Attempting to uninstall r10k-1.4.0
Successfully uninstalled r10k-1.4.0
Attempting to uninstall puppet_forge-1.0.3
Successfully uninstalled puppet_forge-1.0.3
Attempting to uninstall psych-2.0.0
Unable to uninstall psych-2.0.0:
	Gem::InstallError: psych is not installed in GEM_HOME, try:
	gem uninstall -i /usr/share/gems psych
Attempting to uninstall psych-2.0.6
Successfully uninstalled psych-2.0.6
Attempting to uninstall psych-2.0.8
Successfully uninstalled psych-2.0.8
Attempting to uninstall psych-2.0.9
Successfully uninstalled psych-2.0.9
Attempting to uninstall net-ssh-2.9.1
Successfully uninstalled net-ssh-2.9.1
Attempting to uninstall minitest-5.4.3
Successfully uninstalled minitest-5.4.3
Attempting to uninstall kgio-2.9.2
Successfully uninstalled kgio-2.9.2
Attempting to uninstall json_pure-1.8.1
Successfully uninstalled json_pure-1.8.1
Attempting to uninstall json-1.7.7
Unable to uninstall json-1.7.7:
	Gem::InstallError: json is not installed in GEM_HOME, try:
	gem uninstall -i /usr/share/gems json
Attempting to uninstall her-0.7.2
Successfully uninstalled her-0.7.2
Attempting to uninstall git-1.2.8
Successfully uninstalled git-1.2.8
Attempting to uninstall git-1.2.9
Successfully uninstalled git-1.2.9
Attempting to uninstall faraday-0.9.0
Successfully uninstalled faraday-0.9.0
Attempting to uninstall ethon-0.7.1
Successfully uninstalled ethon-0.7.1
Attempting to uninstall cri-2.5.0
Successfully uninstalled cri-2.5.0
Attempting to uninstall bundler-1.7.7
Successfully uninstalled bundler-1.7.7
Attempting to uninstall bundler-1.7.8
Successfully uninstalled bundler-1.7.8
Attempting to uninstall bundler-1.7.9
Successfully uninstalled bundler-1.7.9
Attempting to uninstall bundler-1.7.11
Successfully uninstalled bundler-1.7.11
Attempting to uninstall bigdecimal-1.2.0
Unable to uninstall bigdecimal-1.2.0:
	Gem::InstallError: bigdecimal is not installed in GEM_HOME, try:
	gem uninstall -i /usr/share/gems bigdecimal
Attempting to uninstall minitest-5.5.0
Successfully uninstalled minitest-5.5.0
Attempting to uninstall json-1.8.1
Successfully uninstalled json-1.8.1
Attempting to uninstall i18n-0.6.11
Successfully uninstalled i18n-0.6.11
Attempting to uninstall rdoc-4.0.0
Unable to uninstall rdoc-4.0.0:
	Gem::InstallError: rdoc is not installed in GEM_HOME, try:
	gem uninstall -i /usr/share/gems rdoc
Attempting to uninstall psych-2.0.0
Unable to uninstall psych-2.0.0:
	Gem::InstallError: psych is not installed in GEM_HOME, try:
	gem uninstall -i /usr/share/gems psych
Attempting to uninstall bigdecimal-1.2.0
Unable to uninstall bigdecimal-1.2.0:
	Gem::InstallError: bigdecimal is not installed in GEM_HOME, try:
	gem uninstall -i /usr/share/gems bigdecimal
Attempting to uninstall activemodel-4.1.8
Successfully uninstalled activemodel-4.1.8
Attempting to uninstall json-1.7.7
Unable to uninstall json-1.7.7:
	Gem::InstallError: json is not installed in GEM_HOME, try:
	gem uninstall -i /usr/share/gems json
Attempting to uninstall rdoc-4.0.0
Unable to uninstall rdoc-4.0.0:
	Gem::InstallError: rdoc is not installed in GEM_HOME, try:
	gem uninstall -i /usr/share/gems rdoc
Attempting to uninstall psych-2.0.0
Unable to uninstall psych-2.0.0:
	Gem::InstallError: psych is not installed in GEM_HOME, try:
	gem uninstall -i /usr/share/gems psych
Attempting to uninstall bigdecimal-1.2.0
Unable to uninstall bigdecimal-1.2.0:
	Gem::InstallError: bigdecimal is not installed in GEM_HOME, try:
	gem uninstall -i /usr/share/gems bigdecimal
Attempting to uninstall activesupport-4.1.8
Successfully uninstalled activesupport-4.1.8
Attempting to uninstall json-1.7.7
Unable to uninstall json-1.7.7:
	Gem::InstallError: json is not installed in GEM_HOME, try:
	gem uninstall -i /usr/share/gems json
Attempting to uninstall rdoc-4.0.0
Unable to uninstall rdoc-4.0.0:
	Gem::InstallError: rdoc is not installed in GEM_HOME, try:
	gem uninstall -i /usr/share/gems rdoc
Attempting to uninstall psych-2.0.0
Unable to uninstall psych-2.0.0:
	Gem::InstallError: psych is not installed in GEM_HOME, try:
	gem uninstall -i /usr/share/gems psych
Attempting to uninstall json-1.7.7
Unable to uninstall json-1.7.7:
	Gem::InstallError: json is not installed in GEM_HOME, try:
	gem uninstall -i /usr/share/gems json
Attempting to uninstall bigdecimal-1.2.0
Unable to uninstall bigdecimal-1.2.0:
	Gem::InstallError: bigdecimal is not installed in GEM_HOME, try:
	gem uninstall -i /usr/share/gems bigdecimal
Attempting to uninstall rdoc-4.0.0
Unable to uninstall rdoc-4.0.0:
	Gem::InstallError: rdoc is not installed in GEM_HOME, try:
	gem uninstall -i /usr/share/gems rdoc
Attempting to uninstall psych-2.0.0
Unable to uninstall psych-2.0.0:
	Gem::InstallError: psych is not installed in GEM_HOME, try:
	gem uninstall -i /usr/share/gems psych
Attempting to uninstall json-1.7.7
Unable to uninstall json-1.7.7:
	Gem::InstallError: json is not installed in GEM_HOME, try:
	gem uninstall -i /usr/share/gems json
Attempting to uninstall bigdecimal-1.2.0
Unable to uninstall bigdecimal-1.2.0:
	Gem::InstallError: bigdecimal is not installed in GEM_HOME, try:
	gem uninstall -i /usr/share/gems bigdecimal
Clean Up Complete
root@puppet ~ #

Gems die hier nicht deinstalliert werden können wurden über die Paketverwaltung der Distribution installiert.

Posted in General, Linux, Short Tips | Leave a comment

Wochenrückblick

Das Projekt “30 Posts in 30 Tagen” neigt sich dem Ende zu. An den meisten Tagen habe ich es geschafft – leider aufgrund von Weiterbildung und co – meistens nur sehr kurze Artikel zu schreiben.

Vorhin gab es wieder das tolle Problem “entwickle doch mal unter Ruby 2.1 Code der dann später auf 1.8.7 laufen muss”. Meine Reaktion….

Codeauszug:
Prüfe ob ein Verzeichniss existiert:

root@puppet ~ $ irb
irb(main):001:0> Dir.exists?("/home")
=> true
irb(main):002:0> 

Das war die Entwicklungsumgebung. Live:

root@dns01 ~ $ irb
irb(main):001:0> Dir.exists?("/home")
NoMethodError: undefined method `exists?' for Dir:Class
        from (irb):1
        from :0
irb(main):002:0> Dir.exist?("/home")
NoMethodError: undefined method `exist?' for Dir:Class
        from (irb):2
        from :0
irb(main):003:0> File.exist?("/home")
=> true
irb(main):004:0> File.exists?("/home")
=> true
irb(main):005:0> File.exists?("/home") && File.dir?("/home")
NoMethodError: undefined method `dir?' for File:Class
        from (irb):5
        from :0
irb(main):006:0> File.directory?('/home')
=> true
irb(main):007:0> File.exists?("/home") && File.directory?('/home')
=> true
irb(main):008:0>

Man sieht hier sehr gut die verschiedenen Trial and Error Ansätze. Die passende Idee gab ein Nutzer aus #r10k Inkl. Hinweis auf DevDocs wo wir die passende Doku gefunden haben.

Posted in General, Internet found pieces, Linux | Leave a comment

Dataflow Diagramme

Ein Kollege hat ein hübsches Diagramm gefunden über das fixen von einem kaputten Git Repo. Ein anderer Freund hat mich auf das Dataflow Diagramm des Linux Packetfilter hingewiesen.

Kann man beides mit Sicherheit mal gebrauchen :)

Posted in General, Internet found pieces, Linux, Short Tips | Leave a comment

Leseempfehlung

Ben Roberts hat einen interessanten Artikel über alternatives Deployment von Puppet geschrieben. Ihn, wie auch mich, findet man aktuell in #r10k auf freenode :)

Posted in General, Internet found pieces, Linux, Nerd Stuff, Puppet | Leave a comment

Workaround für yumrepo Resource

Vor einiger Zeit ist mir ein Bug in einem Der Puppet Provider aufgefallen, yumrepo. Es gibt dort den Parameter target, laut Doku kann man hier die Zieldatei angeben in der das Repo definiert wird. Der Name des Repos wird über den Parameter name gesetzt. Leider wird target immer ignoriert, stattdessen wird der Name als Dateiname genommen. Das resultiert in dem Problem das man nur ein Repo pro Datei haben kann. Puppetlabs bietet puppetlabs-release-el-*.noarch.rpm an, diese rpms erzeugen mehrere Repositories welche die offiziellen Puppetlabs Mirror nutzen – allerdings in nur einer Datei. Somit lassen sich die von Puppetlabs eingerichteten Repositories nicht mit den Puppetlabs eigenen Tools (in gleicher Art und Weise) erstellen bzw nachträglich verändern. Dazu habe ich heute einen kleinen Fix in einem Puppetmodul geschrieben. Dies ist noch nicht sonderlich hübsch, funktioniert allerdings sehr gut (auch wenn die RSpec Tests anderes suggerieren). Langfristig werde ich daraus ein eigenes Modul bauen das einen defined resource type bereitstellt welcher yumrepo ersetzt.

Posted in General, Linux, Puppet | Leave a comment

PuppetDB nun mit SSL Protokollwahl

Puppetlabs hat ihr PuppetDB Puppetmodul geupdatet auf Version 4.1.0. Hier gibt es nun den Parameter ssl_protocols, mit dem man verschiedene Protokolle auswählen kann. Leider wurde vergessen eine Möglichkeit zu implementieren diesen Parameter auch zu sezten. Dazu haben ich einen Pull Request erstellt.

Schick wäre es noch wenn man die einzelnen Cipher wählen könnte, eventuell kommt dies in einer späteren Version des Puppet Modules (oder ich bau einen weiteren PR). Die PuppetDB selbst kann dies bereits.

Posted in General, Linux, Puppet | Leave a comment

Linux Short Tip

Tollen CSR erstellen:

openssl req -new -newkey rsa:8192 -days 365 -nodes -keyout mail01.bla.de.key -out mail01.bla.de.csr -sha512
Posted in General, Linux, Short Tips | Leave a comment

duply ging Kaputt

Ich kämpfe aktuell mit einem kaputten Duply Backup auf all meinen Servern:

ssh: Disconnect (code 11): data stalled timeout reached
sftp put of /tmp/duplicity-lCJLzG-tempdir/mktemp-UYqHIH-2 (as duplicity-full.20141117T030008Z.vol1.difftar.gpg) failed: (Try 1 of 5) Will retry in 10 seconds. sftp put of /tmp/duplicity-lCJLzG-tempdir/mktemp-UYqHIH-2 (as duplicity-full.20141117T030008Z.vol1.difftar.gpg) failed: Socket is closed (Try 2 of 5) Will retry in 10 seconds. sftp put of /tmp/duplicity-lCJLzG-tempdir/mktemp-UYqHIH-2 (as duplicity-full.20141117T030008Z.vol1.difftar.gpg) failed: Socket is closed (Try 3 of 5) Will retry in 10 seconds. sftp put of /tmp/duplicity-lCJLzG-tempdir/mktemp-UYqHIH-2 (as duplicity-full.20141117T030008Z.vol1.difftar.gpg) failed: Socket is closed (Try 4 of 5) Will retry in 10 seconds.
sftp put of /tmp/duplicity-lCJLzG-tempdir/mktemp-UYqHIH-2 (as duplicity-full.20141117T030008Z.vol1.difftar.gpg) failed: Socket is closed (Try 5 of 5) Will retry in 10 seconds.

Woher das Problem genau kommt kann ich noch nicht nachvollziehen. Laut ISP ist beim Backupspace alles in Ordnung. Fehler treten auf allen Servern reproduzierbar zu jeder Uhrzeit auf. Aktuell teste ich den Upload via FTP, eventuell klappt dies besser als SCP.

Posted in General, Linux | Leave a comment

Puppet Flow Chart

Puppetlabs hat ein Flowchart über den Ablauf eines Agent Runs erstellt. Hier eine svg meiner Version inkl PuppetDB Anbindung, nginx Loadbalancer und Baremetall Deployment über eine eigene API + PXE und das kostenlose Hetzner installimage. Meine Quelldatein findet man hier.

Posted in General, Internet found pieces, Linux, Nerd Stuff, Puppet | Leave a comment

PDF Sammlung KW46

Obligatorisch zum Freitag:Scaling DevOps with Puppet and Perforce

Posted in General, Internet found pieces, Linux, Puppet | Leave a comment