Wiki Contributions Week 16 - Final

Autoevaluation

Starting the semester, everyone was joining groups which would be working on different things that would help to build the project. I joined the CUDA group, not knowing that I needed a CUDA enabled GPU, which I obsiously don't have. So I started trying to help there, looking for good sources and information which could help the others in the group, but I kinda felt like I wasn't doing much, so I tried another alternatives. Then I started to program some MPI examples, and others stuff, just to get a feel of how MPI programming works, but not joining another group in the process. There I learned some interesting stuff using mpi4py, but still not doing a great progress.

Finally, somehow I ended with the cluster group, but I didn't know anything about what they were doing, and I was supposed to do a Web Interface for the Cluster in Ruby, which resulted in a failure because I couldn't do any notable progress, which is entirely my fault, and because of that, others had to do my part.
In conclusion, I didn't do anything to add to the cluster.Period.

Nominations

Juan Carlos, Rafael, and Roberto the only ones with own initiative.

Wiki Contributions - Week 15

For this week, being the topic Benchmarking, I investigated something about that, and found a pretty interesting piece of code that does exactly a benchmark of a certain computer using processes and threads and running normally, to find which ones are more efficient in certain circumstances, like executing a simple mathematical  operation or doing an I/O operation. The results were horrible, in my case, because my computer is not that good, but analyzing the information it is seen that the operation  runs "faster"(or more like less slower) most of the time using processes.

Link to the wiki: 

• Benchmarking

Nominaciones:

Juan Carlos, Victor y Saúl

Wiki Contributions - Week 14

For this week, what I wanted to do is to issue some commands from a web page in ruby on rails, to do some things through ssh on a virtual computer. I managed to do this, but not as I expected, because I can't actually get to understand how ruby on rails works, and the only thing I could do is put my code that runs an ls command through ssh, somewhere between other operations of the web application. The code runs, and effectively does a ssh, but obviously I wanted the web application to ask for a command to run, which I couldn't done.

Anyway the specific code that I added to a ruby on rails application is the following:

#!/usr/bin/env ruby
require 'rubygems'
require 'net/ssh'

HOST = '192.168.1.75'
USER = 'user'
PASS = 'its'

Net::SSH.start( HOST, USER, :password => PASS ) do|ssh|
  output = ssh.exec!('ls')
  puts output
end

The code uses the Net::SSH gem, which I explained previously and there is an entry about it on the wiki.
This is not exactly the code itself, because I had to adapt it to the ruby on rails application, which works pretty weird and the only thing I could do to print the output data from the remote ls instruction, was to use some kind of html function that ruby on rails added to the code.

I updated the wiki with this and other examples I found using Net::SSH.

• Net::SSH Examples

Nominaciones:


Alex and Victor

Wiki Contributions - Week 13

More than a contribution to the wiki, this week I have to do a report of what I've been doing. First,  I finally could connect the nodes of the virtual guests(VirtualBox) with the host, so I could do ping, ssh and other stuff. This will be useful in order to test the web interface, which I've been working on too. Now as none of this is new, I'll just add the new information to the already existing entries in the wiki, and publish some screen captures.

Link to the wiki

• SSH(updated with installaton and use)

Some advances screen captures:

Ping

             

SSH

Future contributions

With the guests and host connected, now I can proceed to write some code in the web interface to do some basic stuff, like ssh, send files, copy files, etc. to these guests.

Wiki Contributions - Week 12

For this week I went back into the ruby on rails web interface, so looking into the documentation and some other information I found a library called Net::SSH, used to interact between Ruby and SSH. This will be needed in order for the web interface to use ssh to send the tasks to the nodes, and to receive the output from each node, in order to keep track of the cluster performance.


Links to the wiki:

• SSH
• Net::SSH

Nominations:

Juan Carlos

Wiki Contributions - Week 11

Researching about clusters and administraton of clusters, I found a good application that will probably help a lot  with the cluster. ClusterSSH, which in short helps to create  a ssh connection with multiple hosts and run commands in all of them. This could be extremely useful, because the web interface could use this application instead of basic ssh to do some tasks in the hosts. Also I managed to mount some Ubuntu 10.10 (32 bit) virtual machines using VirtualBox, which I'll use to make tests:

Unfortunately, I still couldn't create a network in order for the nodes to connect, but that should be easy enough to do it in a couple hours, I'll look into that in the following days.

Class links:
- ClusterSSH




And for laboratory:
- High Availability Clusters and Fault Tolerance


Extras for laboratory(Guessing game, both game's words are in the same post):
- Extra Laboratory

Wiki Contributions - Week 10

For week 9 I didn't get any good advance because I was kind of busy with some other stuff, I just managed to read some Ruby on Rails tutorials to some functions, but nothing big. So leaving that aside for a while, I decided to investigate a little more about the beowulf clusters, and more particularly into the master-head node, to see how the master node is supposed to work, and what should the master node do in the network. Then I found out that there is a way to create a virtual cluster, using VirtualBox which allows you to build and test the cluster without the need for the extra hardware.This could be useful to test some stuff first, before using the Cluster, but only for that, because the performance will be probably poor. Of course I still will be working with the web interface, with the help of Raúl I hope.

For the class, the wiki is crashed currently(April 15th) but again, I'll post the information here in my blog first, in order to upload it later to the wiki.


-- Edit -- 


Now the wiki is online again, so here it is the direct link to the wiki:

- VirtualBox

And for the laboratory, a small investigation about some MPI implementations:

- MPI, OpenMPI and MPICH

Wiki Contributions - Week 8

For this week, I was appointed with a new and interesting task. To do a web interface using Ruby on Rails. This web interface would be used to distribute tasks between the nodes of a cluster, check the state of the nodes, and some other stuff. So to do this, I first installed Ruby on Rails and started testing and reading about it.

So to contribute in the wiki, this week I'll add a tutorial of how to install ruby on rails and all the needed stuff there. And I'll be uploading code to something like Github to keep the code online and updated. Also if I found something interesting about Ruby On Rails I'll upload it in the wiki as well.

Anyway, the link to the wiki is the following:
 - Ruby On Rails

For the laboratory Ruby On Rails Installation and Demo
-Demo App
Nominations:

Roberto and Juan Carlos and Gabriela for their great planning of the future tasks.

Wiki Contributions - Week 6

In this week, I finally understood the basics of MPI. Reading examples and modifying them I could make my own MPI programs. Right now I just can do simple stuff, like sending and receiving data, or running parallel stuff, but I'll try to understand more complex things in the near future.

As of today, March 11, the server is down, so I can't upload anything to the wiki right now, but I'll post it in my blog in the meantime, but I'll put it in the wiki when it is available.

The information to the wiki is the following:

• Message Passing Interface (From the Beginner's Guide to MPI)
• Also some code I was going to upload to the wiki, but that's on the laboratory post anyway.

For the laboratory I explained on detail the important basic functions used in MPI for Python with some examples.

Future contributions


I'll keep working with more complex things with MPI, but I also want to find out other things in which I can help, but I think I'll find out about that on the meeting.

Nominations

Rafael, for his initiative of joining all of us in a meeting. I plan to attend, providing I have money for wherever they want to met.

Message Passing Interface (MPI)

MPI

Message Passing Interface (MPI) is a standardized and portable message-passing system designed by a group of researchers from academia and industry to function on a wide variety of parallel computers. The standard defines the syntax and semantics of a core of library routines useful to a wide range of users writing portable message-passing programs.

Functions

MPI standard has many functions in it, approximately 125. However, many of the advanced routines represent functionality that can be ignored until one pursues added flexibility (data types), robustness (nonblocking send/receive), efficiency (``ready mode"), modularity (groups, communicators), or convenience (collective operations, topologies). MPI is said to be small because there are six indispensable functions from which many useful and efficient programs can be written.

The six functions are:

1. MPI_Init(Omitted in python) - Initialize MPI 
2. MPI_Comm_size - Find out how many processes there are 
3. MPI_Comm_rank - Find out which process I am 
4. MPI_Send - Send a message 
5. MPI_Recv - Receive a message 
6. MPI_Finalize(Omitted in python) - Terminate MPI

Explanation

MPI_Init. The call to MPI_Init is required in every MPI program and must be the first MPI call. It establishes the MPI execution environment.

 int MPI_Init(int *argc, char ***argv)

 Input:
       argc - Pointer to the number of arguments
       argv - Pointer to the argument vector

MPI_Comm_size. This routine determines the size (i.e., number of processes) of the group associated with the communicator given as an argument.

 int MPI_Comm_size(MPI_Comm comm, int *size)

 Input:
       comm - communicator (handle)
 Ouput:
       size - number of processes in the group of comm

MPI_Comm_rank. The routine determines the rank (i.e., which process number am I?) of the calling process in the communicator.

 int MPI_Comm_rank(MPI_Comm comm, int *rank)

 Input:
       comm - communicator (handle)
 Output:
       rank - rank of the calling process in the group of comm (integer)

MPI_Send. This routine performs a basic send; this routine may block until the message is received, depending on the specific implementation of MPI.

 int MPI_Send(void* buf, int count, MPI_Datatype datatype, int dest,
              int tag, MPI_Comm comm)

 Input:
      buf  - initial address of send buffer (choice)
    count - number of elements in send buffer (nonnegative integer) 
    datatype - datatype of each send buffer element (handle)
      dest - rank of destination (integer)
      tag  - message tag (integer)
      comm - communicator (handle)

MPI_Recv. This routine performs a basic receive.

 int MPI_Recv(void* buf, int count, MPI_Datatype datatype, int source,
              int tag, MPI_Comm comm, MPI_Status *status)

 Output:
      buf  - initial address of receive buffer 
    status - status object, provides information about message received;
          status is a structure of type MPI_Status, the element
          status.MPI_SOURCE is the source of the message received, 
          and the element status.MPI_TAG is the tag value.
          
 Input:
    count - maximum number of elements in receive buffer (integer)
    datatype - datatype of each receive buffer element (handle)
    source - rank of source (integer)
    tag  - message tag (integer)

    comm - communicator (handle)

MPI_Finalize. This routine terminates the MPI execution environment; all processes must call this routine before exiting.

 int MPI_Finalize(void)

MPI_Bcast. This routine broadcasts data from the process with rank "root" to all other processes of the group.

 int MPI_Bcast(void* buffer, int count, MPI_Datatype datatype, int root,
               MPI_Comm comm)

 Input/Output:
    buffer - starting address of buffer (choice)
    count - number of entries in buffer (integer)
    datatype - data type of buffer (handle)
    root - rank of broadcast root (integer)
      comm - communicator (handle)

MPI_Reduce. This routine combines values on all processes into a single value using the operation defined by the parameter op.

 int MPI_Reduce(void* sendbuf, void* recvbuf, int count, MPI_Datatype
                datatype, MPI_Op op, int root, MPI_Comm comm)

 Input:
    sendbuf - address of send buffer (choice)
    count - number of elements in send buffer (integer)
    datatype - data type of elements of send buffer (handle)
    op - reduce operation (handle) (user can create using MPI_Op_create
          or use predefined operations MPI_MAX, MPI_MIN, MPI_PROD, MPI_SUM,
          MPI_LAND, MPI_LOR, MPI_LXOR, MPI_BAND, MPI_BOR, MPI_BXOR,
          MPI_MAXLOC, MPI_MINLOC in place of MPI_Op op.
    root - rank of root process (integer)
    comm - communicator (handle)

 Output:
    recvbuf - address of receive buffer (choice, significant only at root )

References:

http://www.eecis.udel.edu/~saunders/courses/372/01f/manual/manual.html

Wiki Contributions - Week 5

For this week, I searched a little about sockets in order to program a client-server instance of mergesort in python. At first the script just accepted a single client, which wasn't any good, but later on after reading a little bit about sockets I realized the problem was just that, I had the accept routine before the server script entered the loop, so it wouldn't accept any more clients.

Anyway the script is in the wiki, I tried to comment it the most I could so it would be understandable, but it is not very difficult to understand

• Client-Server Mergesort
• Sockets(from the Beej's Guide to Networking, Excelent Guide)[Still not on the wik]

For the laboratory, I made this post explaining personally my python client-server mergesort:

• http://synnick.blogspot.com/2012/02/client-server-mergesort.html

Future Contributions

I've been also working with MPI for Python but until now I don't have any good results to share, so I will keep on trying with that.

Nominations


- Juan Carlos and Cecilia

Wiki Contributions - Week 4

For this week in the class, I installed all the tools needed to compile and run MPI codes using python as a programming languages. I had issues with that, because python wasn't updated, and some other stuff was needed in order to build and install packages, so I actually didn't had a chance to program with it this week, but I'll do it.

The links to the wiki:

• MPI for Python Install-Compile guide
• Examples with MPI for Python

Future contributions

• I'm going to do something with MPI, probably for the next week. My idea is to do something like merge sort, or anything that sends some data in order for the server to do something with it.

Nominations

- Isaias: Mergesort in CUDA
- Rafa and Juan Carlos: MPI with C-C++, with good examples

Wiki Contributions - Week 3

In this week, I investigated a little about Cuda C syntax, I know it's pretty much C, but it contains some new stuff that I think needed to be explained, so it would be easier to pass other language programs to CUDA in order to run them in the GPU. Also it's pretty much the only thing I can do with the GPU team, because I don't have a CUDA enabled GPU, anyway I'm planning to also do parallel programming stuff in other languages(python, java, [also I'm particularly interested a lot in Perl now because of this, I want to do something similar, but legal]).

Anyway my contributions to the wiki for the class were the following:

• CUDA Abstractions(Kernels, Threads[Grids and Blocks], Memory, Host and Device) with code, and CUDA Language Extensions, Function Type qualifiers, variable qualifiers, etc.
• Cuda C compiling guide in Linux, this is a small guide for compiling CUDA code (which doesn't require Nvidia graphic cards). So, because it is a guide and not information itself, I think it's better to post it here in my blog, and put a link in the wiki just in case somebody need it.

And in the laboratory:

• Parallel Programming in Perl scripts

Future contributions

And what I'm planning to do in a near future:
- Compile CUDA C code, to learn about common mistakes and what can I do. The area in which I can help is to learn the much I can the language extensions and other CUDA C stuff in case we need it.
- Parallel scipt in some language to copy something from webpages, like the Pirate Bay one, but legal.

Nomination

My nomination of this week is again Isaias, because he told me that I can compile CUDA C code in my computer(but not run it), and also he made some CUDA C code about sum of arrays which is a pretty good advance.

Compiling Cuda C code in Linux

If you don't know how to do it, here I'll explain in short how do you install the CUDA toolkit and what do you need in order to compile CUDA C code.

First of all, this is just to COMPILE, not to run, I can't do that part of the guide because I don't own a Nvidia graphics card or whatsoever, and of course I'm not going to post someone else's guide to that if I can't test if it works. Knowing that lets begin.

First, download the CUDA Toolkit 4.1 from here. I downloded the Ubuntu 10.04 version because I have Ubuntu 10.10 which is an earlier version and I didn't had any problem with it. It weights around 189 MB so be patient while it downloads.

When the download finishes, run the downloaded package. It should be called something like cudatoolkit_4.1.28_linux_ubuntu10.04.run(depending on the version you download, and your OS this will change obviously), give it executing permissions using chmod a+x cudatoolkit_4.1.28_linux_ubuntu10.04.run and run it with ./_ubuntu10.04.run.

If you can't, use sudo su and type your password to identify yourself as root, and run it again. This should produce an output like this:

As we can see, it will ask you the install directory, if we don't really care just press enter and it will be installed on /usr/local/cuda.

When it finishes we will probably need to set the PATH to this directory, or else we will not be able to compile our code, to check if your PATH is set try using the nvcc command, if you don't have the PATH set, the terminal will output something like "No command 'nvcc' found...", in that case we will need to set the PATH using: export PATH=$PATH:/usr/local/cuda/bin. Like this:

As we can see at first it didn't recognize the nvcc command, but after setting the PATH the nvcc command was recognized.

Now we can try to compile our own code in CUDA C, for this you can use the following Hello World example:

Try to compile it with nvcc filename.cu(notice the .cu extension). It should just produce a file named filename.cu.cpp.ii. Like this:

If you get the "Error trying to exec 'cc1plus'" message, install g++ and that should fix it.

As a compiler, if the code has syntax errors, or something like that, it shows you the line and the type of error the code has. For example making a simple ";" missing error produces the following feedback at compiling.(Which in this case doesn't tell anything about the ";" missing)

And that is all about compiling CUDA, for other weekly reports in class, I will probably try to compile some Merge Sort, Bubble Sort, or something like that in CUDA and I will ask someone else to run it for me.

Wiki Contributions - Week 2

My contributions to the wiki the second week were the following:

-Entry about GPUs and GPU computing in general here.

-Added a C and CUDA C code comparison here.

I'll upload the image here to link it in the wiki:

Image taken from the following pdf:

http://cscads.rice.edu/workshops/july2007/autotune-slides-07/RichardJohson.pdf

Wiki Contributions - Week 1

Like I mentioned in my laboratory post, I am part of the GPU team, but I don't have a Nvidia graphics card, so I researched for other options.

My contributions to the wiki this week are the following:

• OpenCL

I nominate Isaias for his great post about CUDA, which also includes a guide to check if our graphics card is compatible with CUDA, and an overall guide to install it with images and a pretty good explanation step by step.

AMD APP and OpenCL

For this week I decided to do research on the Internet for other alternatives for GPU Computing. This is because I don't own any computer with a Nvidia graphics card, and of course, I can't run CUDA without it. The best I could do (and probably will do) is to compile code in my own computer, and ask somebody else of my team with a supported graphics card to run it. This would be somewhat problematic, so that is why searched for something similar to CUDA.

I own a laptop and a PC, and both of them have ATI graphics card, so my first idea was to look for something similar to CUDA from AMD. The first thing I found was ATI Stream which was upgraded to AMD APP (Accelerated Parallel Processing). The AMD APP SDK supports OpenCL, which is open and royalty-free programming framework for general-purpose computations on heterogeneous systems.

APP Technology

AMD APP technology is a set of advanced hardware and software technologies that enable AMD graphics processing cores (GPU), working in concert with the systems x86 cores (CPU), to accelerate many applications beyond just graphics. The AMD APP SDK allows you to develop your applications in a high-level language, OpenCL (Open Computing Language).

To see if your ATI graphics card is compatible with the SDK, you can check this link:

http://developer.amd.com/sdks/AMDAPPSDK/pages/DriverCompatibility.aspx#requirements

OpenCL

OpenCL is the first open and royalty-free programming standard for general-purpose computations on heterogeneous systems. OpenCL allows programmers to preserve their expensive source code investment and easily target both multi-core CPUs and the latest GPUs, such as those from AMD and Intel.

Developed in an open standards committee with representatives from major industry vendors, OpenCL gives users what they have been demanding: a cross-vendor, non-proprietary solution for accelerating their applications on their CPU and GPU cores.

Differences between CUDA and OpenCL:

Multiple comparisons have been drawn between CUDA and OpenCL since its inception. They both draw the same conclusions: if the OpenCL implementation is correctly tweaked to suit the target architecture, it performs no worse than CUDA. Since the key feature of OpenCL is portability (via its abstracted memory and execution model) the programmer is not able to directly use GPU specific technologies unlike CUDA. CUDA is more acutely aware of the platform upon which it will be executing, as it is limited to Nvidia hardware, and therefore provides more mature compiler optimisations and execution techniques (such as the use of texture memory which cannot be done in OpenCL). 

A list of some Pros and Cons between CUDA and OpenCL:

So I think I find CUDA be a better option in parallel computing, with the only bad drawback since it is only available to Nvidia GPUs, also the overall performance and documentation seems to be more complete in CUDA, because they just have to focus in their own devices. OpenCL in the other hand works with a more open variety, being Intel, Nvidia, and ATI devices supported.

References:

• http://siroro.co.uk/2011/08/02/gpgpu-programming-languages-compared-opencl-cuda-directcompute/
• http://developer.amd.com/sdks/amdappsdk/pages/default.aspx
• http://www.pcper.com/reviews/Graphics-Cards/ATI-Stream-vs-NVIDIA-CUDA-GPGPU-computing-battle-royale
• http://en.wikipedia.org/wiki/OpenCL
• http://www.khronos.org/opencl/

(For information about CUDA, Isaias made a good post about it)