Tag Archives: rsync

XDA’s Build server for Recognized developers, in the works, does not have a public access section for the file system where files may be uploaded and may be downloaded. That may probably change in the future, as this is a beta thing kept under wraps, and open for only RDs.

For now, if you wish to transfer files from the server, you may use the scp service, either with a frontend like Filezilla, or from the command line. This is important for me because I’d like to remote upload my compiled files from the XDA build server to Goo.im, or Sourceforge and offer it to my users for download.

So on the second remote server (like Goo or Sourceforge) where you have ssh access, follow the following steps:

Login with ssh in the usual way with Putty or ssh to the second server (Goo or Sourceforge, will be called server2) where you have a public section of the file system (public_html or www).

Create a remote key on this server (server2), with:

ssh-keygen -t rsa

Now, you have to transfer the public key for this server2 to the server1. In other words, you are going to remote access server1 from server2. So you have to have a public/private key pair on server2, and transfer its public key to server1.

Once the key pair is set up, transfer the public key to server1. If server1 already has other keys setup, edit ~/.ssh/authorized_keys in a text editor (I prefer emacs), and add the new public key to the last line, and press enter (for a newline char). Save authorized_keys.

Now server1 will accept remote connections from server2.

Transfer from Remote to Local (Or two remotes, if one of them is logged into)

Now, use the following scp command to transfer the file remotely from the XDA filesystem, to the currently logged in server:

scp -i .ssh/goo_xda [email protected]:pathtofile targetonhost2

 

Eg:

 

scp -i .ssh/goo_xda [email protected]:~/myfiles/compiled.tar.bz2 ./myfolder/

 

This will use a ssh key at .ssh/goo_xda (Private key on the logged in server-Server2), to login to server 1 (devserver.xda.com) with username dz, and copy a file at ~/myfiles/compiled.tar.bz2 from server1 to  ./myfolder/compiled.tar.bz2 on server2. Server2 is where you have logged in currently using ssh/putty. Server1 is the XDA build server from which you want to transfer files.

Of course in some instances, you can omit the -i parameter to the private key by adding the key with ssh-add, to the agent. However I found that Goo, XDA and Sourceforge werent allowing this.

 

Transfer from Local to Remote (Or two remotes, if one of them is logged into)

In this example, I am logged into the XDA build server and am copying a file to my Sourceforge remote server.

 

scp download/phpfm_281012_064121.zip [email protected]:/home/users/d/dr/droidzone

Here, I’m copying a file download/phpfm_281012_064121.zip from the XDA server where I am logged in (or from my linux local machine) to the host web.sourceforge.net to the specified directory. I need to prefix my username. If I need a key, I would prefix it with -i option.

So, essentially, what you have to do is this:

Generate a key pair:

-bash-4.1$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/droidzone/.ssh/id_rsa): /home/droidzone/.ssh/id_rsa_xda
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/droidzone/.ssh/id_rsa_xda.
Your public key has been saved in /home/droidzone/.ssh/id_rsa_xda.pub.
The key fingerprint is:
2d:89:29:71:19:1d:bc:33:d0:cd:60:21:77:00:af:8f [email protected]
The key's randomart image is:
+--[ RSA 2048]----+
|      +=BB.      |
|      .B+.o      |
|    . o...       |
|     o ++o       |
|    . + So.      |
|     . o .       |
|      E .        |
|                 |
|                 |
+-----------------+

It doesnt matter where you generate the key files. It could be on your home PC too.

Next, let’s assume you’re going to build on XDA then transfer to Goo.

So. login into XDA (the local server).

Copy the key pair’s private key to XDA, into ~/.ssh/

Change the default permissions to a more secure permission.

chmod go= ~/.ssh/mynewkey

Now, add the public key for this pair to Goo (the remote server)

Edit ~/.ssh/authorized_keys

Go to the end of the last key line. Press enter and paste the new public key.

Save the file.

Now, you can copy the file with the following line:

scp -i ~/.ssh/goo_xda N7100XXALJ3_N7100ODDALJ2_INU.zip [email protected]:/home/droidzone/public_html/firmware/n7100/
N7100XXALJ3_N7100ODDALJ2_INU.zip

This uses the private key ~/.ssh/goo_xda and copies the file N7100XXALJ3_N7100ODDALJ2_INU.zip from the current directory on XDA (where you’re logged into-the local server), into the location /home/droidzone/public_html/firmware/n7100/ on the Goo server (remote). Piece of cake, isnt it?

As you can note, this is extremely useful as XDA does not provide a public_html folder where built files can be provided to users.

Working with Filezilla on XDA’s Build server

Due to the absence of password auth on XDA build server, one needs to setup key based authentication. For this, a third party program named Pageant is required. Another requirement is a key in PPK (Putty) format. The PPK conversion of OpenSSH keys may be done with Puttygen.

Download Pageant from Putty’s site (Choose the x86 version)

Run Pageant.exe (standalone exe)

An icon for pageant appears in the system tray.

Right click it and “Add key”

Choose the ppk key stored for XDA.

Choose Ok.

Start Filezilla, setup the following:

Host: Eg: buildserver.something.xda-developers.com (The actual hostname is provided to XDA RDs)

Port: 22

Protocol: SFTP

Logon type: Normal

Username: [your username]

Password: [leave empty]

Note that if Pageant is working, there is no need to setup key in Filezilla. We just leave an empty password. Pageant takes care of auth.

Transferring large files: rsync

rsync --partial --progress -avvvz -e "ssh -i /home/droidzone/.ssh/xdamachine_key_openssh_pvt" random [email protected]:~/

–partial: Continue interrupted downloads

–progress: Display progress

-a: archive with preserved permissions

-vvv: verbose mode

z: gzip mode transfer

 Rsync exit codes: