Further libssh2 wrapper updates
I’ve written a small shell script that takes the archive downloaded from http://www.libssh2.org, extracts it, compiles it, extracts the headers and archive library and then tidies up after itself. The results just need adding to Xcode and you have a working libssh2 implementation.
Updated source code:
https://github.com/thebsdbox/DFSSHWrapper
Read MoreUpdate to the sshwrapper
Had quite a few emails recently about using the ssh wrapping class I wrote aaaages ago. I’ve traded a couple of emails back and forth.. and decided that it would be easier for everyone if I just updated these old classes.
So the changes:
- Added DFSSHConnectionType, this class is used to define how ssh will attempt to connect (password/key/keyboard)
- Moved everything to a namespace (DF)
- ARC
- Tidied up the code, and sorted an issue with CStrings making a mess when converting to an NSString
- Other things I did ages ago.. (no idea)
It’s uploaded to github.. let me know if there is any problems..
https://github.com/thebsdbox/DFSSHWrapper
[UPDATE]: Added the ability to place a timeout on a command sent over ssh…
Read MoreLVM2 device names and device paths
If you use LVM (Linux volume manager) you’ll be no doubt aware that the ability to place your volumes in groups and name the volumes accordingly makes administration much easier. Having a volume group called oracle and then a binaries and data volume means a tiny glance at a system identifies what is doing what (or so I was under the impression). When it comes to doing any sort of low level administration or system metrics, you may notice that device paths change. Using fdisk or looking at /proc metrics and you’ll suddenly come across dm-0 etc.. devices, which clearly are the lvm devices under another name. Scanning quickly through a ton of web pages and the lvm2 documentation turned up nothing.
Then I realised that the first two columns in /proc/diskstats where the major/minor device numbers, which when compared with the entries in /dev/mapper allowed me to match up the devices. This can also be achieved by looking at the major/minor numbers from lvdisplay and dmsetup ls. However the latter two tools require root privileges so clearly were not going to work with my current project.
Turned out that the simplest method is to look through the sysfs file hierarchy as the block devices can identify other names that they have. The following command will iterate all of the block devices with alternative names and print them in a format that can be piped into anything else, or in my case be read easily into an NSArray.
grep -H "." /sys/devices/virtual/block/*/dm/name \ | sed 's/\/sys\/devices\/virtual\/block\///g' \ | sed 's/\/dm\/name:/ /g'Read More
Getting files through a terminal window
This is a technique I had to use numerous times for a previous job where I would need to transfer files from servers that could not be connected to directly. In the majority of companies there will be numerous networks, where “jump boxes” are required to get across various networks and get to the server in question.
The usual approach of moving files to servers would be through a variety of means (FTP/SCP/NFS/CIFS etc.), however in the case of numerous jump boxes and networks would mean that having to transfer the file between each jump/network. It is possible to use copy and paste however this would only work in the case of text files, trying to display or copy binary data can cause all manner of issue and really mess up your terminal window. So for me the best solution is to UUEncode the binary data into ansi text which can be safely copied out of the terminal window, pasted into a file on your local machine and UUDecoded back to binary data again.
To do this simply UUEncode a file, and the output will be presented to STDOUT i.e. the terminal window.
$ uuencode test.rpm test.rpm
Note: the double typing of the name is required as the first argument is the file to uuencode, whilst the second argument is the name of the file that will be outputted. The output will be presented to STDOUT as shown in an example :
begin 644 test.rpm
M"GP]+2TM+2TM+2T]6R!796(@=G5L;F5R86)I;&ET:65S('1O(&=A:6X@86-C
M97-S('1O('1H92!S>7-T96T@73TM+2TM+2TM+2T]?`I\/2TM+2TM+2TM+2TM
The next step is simply to copy everything from the word “begin” to the word “end” out from the terminal window and then paste it into a text editor of your choice on your local machine under a temporary name. This will then need opening with uudecode, which will then process the text and spit out the file under the filename specified with the encoder.
$ uudecode temporary.uua
In the same location will be the decoded file.
Read MoreESXi v4.1 SFTP access
[UPDATE]
This is a 32bit binary, which I think needs some pretty old kernel version. Hence it only works on 4.0, I will try and get an updated release for 4,1 (*note) ESXi 5 comes with sftp-server already.
I came across something interesting while fiddling earlier, after spending about 2 hours building a static release of openssh server that was going to replace dropbear. I’d gotten to a point where I could build a i386 release of the binaries with no random library requirements and sshd would start and listen on a port defined in /etc/ssh/sshd_config. unfortunately starting ssh in debug mode allowed me to see numerous glibc errors during connections and explain why I couldn’t connect. At this point I don’t think there is any real way of replacing dropbear with a complete openssh solution even statically linking. Even testing the openssh sftp binary that had been compiled showed that it wasn’t coping with a system call not returning UIDs correctly meaning that it would report a FATAL error and close continually.
Given openssh wasn’t going to be replaced I researched about dropbear and if there was a newer version perhaps with sftp, unfortunately not. Eventually I came across notes on a blog mentioning that dropbear “supports” openssh sftp. After restoring ESXi back to its default filesystem settings (ssh enabled) it appears the attempting to sftp to esxi returns the following error.
After compiling a slightly older version of openssh (static) I found a release of sftp-server that will once placed in /sbin on ESXi allows full usage of sftp (including sshfs mounting) binary below.
Read More
File access in ESXi 4.1 (nfs and tcp)
I’ve had numerous occasions were i’ve needed to upload files to the actual file systems on an esxi system, the only ‘proper’ method is using the abysmal virtual infrastructure client and working mainly on a mac means I need to use VMware Fusion for windows to run the client to connect to the server (overkill). So it’s possible to enable ssh access to the server using the tech support menu, which allows access to the underlying hypervisor and it’s file systems and therefore it’s possible to scp files to the filesystems again this is quite slow and overkill due to the encryption being used. Also due to dropbear being used for the ssh it doesn’t use sftp, which means that you can’t mount the filesystems ala. FUSE and sshfs.
I should say at this point, the goal of all this was to allow me to keep all my ISOs on one server and be able to access them from everywhere also, I wanted a PXE server to be able to access the ISOs and loopback mount them and then present the contents via NFS to the installers started by PXE.
So looking around I found some ftp binaries that should work on ESXi, given that the console access for ESXi is done with busybox there is no file command to determine what binary type the files are so I was unaware of what binaries I could run inside ESXi. This all worked fine following the instructions located on the vm-help.com website here however a few of the instructions are a little bit incorrect such as the path to tcpd is incorrect in inetd, however i’ll leave you to fix that. So on the PXE server using FUSE again and curlftpfs to mount the filesystem and this revealed a glaring bug as soon as I loop back mounted the first ISO. Unfortunately the problem lies in the fact that curlftpfs will use memory to store the file as it downloads it for access by FUSE, so trying to open a 4GB DVD ISO quickly exhausted my PXE servers memory and then it became unresponsive, great.
Further research turned up a blog post about some guy trying to use unfs to enable nfs sharing between two ESXi boxes, more specifically it was mentioned that linux binaries would work fine in the ESXi service console. One thing that was slightly confusing was that ESXi is x86_64 (64bit) however binaries that you need for the service console have to be 32bit otherwise you’ll get a confusing error that the binaries can’t be found when you try and run them due to busybox’s odd handling of errors. I present below the binaries required for nfs in ESXi :-
These are pretty easy to use, scp the file over to ESXi and untar it in /tmp al that’s left is to place the files in /tmp/sbin into /sbin and the files in /tmp/etc into /etc. The /etc/exports contains one entry to give access to /vmfs/volumes, which means that accessing the nfs share will give you the UUID paths for the disks containing VM’s and ISOs. To start the nfs server, start portmap first and then start unfsd which should be started the following way (unfsd -d &), this is due to unfsd not being able to disconnect from console on start up (something to do with busybox I assume).
One final note, is that once another machine connect to the nfs share portmap will go start using 50%-70% cpu and will need stopping and starting for other nfs clients. I’m still looking into this, however having a cron job to restart the process every few minutes should do the job.
Read More