Monday, May 21, 2007

Problems changing a BOINC server's name

The following email summarizes a problem that we had when we changed our server's name to make it available outside our firewall.
Solution found:
I wrote:

Okay we changed the name of our server from an internal name to an external one that can be accessed across the Internet. My problem is that bin/start reports:

Entering ENABLED mode
Starting daemons

And nothing happens. "ps aux|grep feeder" only reports the grep instance.

I have changed the server name in the mysql.user table, in /etc/sysconfig/network, in config.xml and I'm sure a few other locations. Restarted the machine, etc. I also verified the permissions on the new log_... and pid_... folders.
Found the problem. The 'stop_daemons' file was not being deleted by bin/start. I did not realize that the config.xml host setting should only be 'www'. Not the FQDN.

The config host has to match the localhost name that is internal to bin/start. This is *not* the same as what is returned by 'hostname'.

This took awhile to find, because no log files were being generated, neither did start report any errors. I only saw it by running feeder under gdb, where the output apparently goes to stdout/stderr instead of (or in addition to) the log file.

--
Ron Parker
UPDATE:
One additional file that needed changed in the SciLINC project directory was html/user/schedulers.txt.

It is also necessary to update the app_version table. The path to the application files on the server is hard coded in the XML data fields of this table. Running bin/update_versions will update it, but it may be necessary to first delete the rows that refer to the old server.

Friday, January 19, 2007

Visual Studio 2005 missing Msi.lib

Visual Studio 2005 only installs msi.lib for the AMD-64 platform. Installing the latest Platform SDK, PSDK will install it for x86, IA-64 and AMD-64. As of this post, Microsoft Platform SDK for Windows Server 2003 R2 was the most recent version. That page should say whether or not there is a more recent version.

Once it is installed, you have to tell Visual Studio where to find the PSDK files. The file path settings are part of the Visual Studio options which are accessed from the ToolsOptions... menu item.

Under Projects and Solutions, select VC++ Directories. Then, pick each item under Show directories for looking for paths have PlatformSDK in them. Insert an entry just before it that refers to the same subdirectory under $(ProgramFiles)\Microsoft Platform SDK for Windows Server 2003 R2.

Visual Studio Options

The up- and down-arrow buttons may be used to move the new entry to the right place in the list.

Now, Rebuild any solution that needs the msi.lib library.

Tuesday, January 16, 2007

Importing Python Subversion Archive into Git

While it is not well advertised, it is possible to download a snapshot of the Python Subversion repository via rsync.
rsync -avP svn.python.org::ftp/pub/svn.python.org/snapshots/projects-svn-tarball.tar.bz2 .
Note: I discovered later that it can be downloaded by browser as well, http://svn.python.org/snapshots.

Once the file is downloaded expand it:
bunzip2 projects-svn-tarballs.tar.bz2|tar xf -
Then use git-svnimport to create the baseline git repository:
git-svnimport -v -C python.git -T python/trunk -t python/tags -b python/branches file://`pwd`/projects
Following this the archive may be brought up to date and kept upto date using:
git-svnimport -v -C python.git -T python/trunk -t python/tags -b python/branches http://svn.python.org/projects

Thursday, January 11, 2007

Importing SourceForge Subversion Projects into Git

At least for the initial import it is faster to create a local mirror of the desired repository using rsync. Taking the Simplified Wrapper and Interface Generator project, SWIG, as an example:

rsync -avz rsync://swig.svn.sourceforge.net/svn/swig/* swig.svn
Made a copy of the Subversion SWIG repository in about 10 minutes.

(On CentOS 4, I had to upgrade svn from the source to be able to access this mirror. I installed the RHEL4 RPMs that were pointed to on the Subversion download page. I needed the apr, apr-util, subversion and subversion-perl packages.)

Then I was able to import this into a git repository using:

git-svnimport -C swig.git \
file://`pwd`/swig.svn
This took about 13 minutes.

After importing a new repository, repacking recommended:

cd swig.git
git repack -a -d
This reduces the amount of space the repository requires. It can be significant on a large or active repository.

Future updates can then be done fairly effeciently from within the swig.git subdirectory with:
git-svnimport -C . https://svn.sourceforge.net/svnroot/swig
Because it is no longer needed, the swig.svn directory can now be deleted.

Just some statistics
While the rsync'd Subversion repository took up 238MiB, the git repository only used 63MiB. Most of this is the working copy of the code. The actual repository swig.git/.git only occupied 25MiB. (Barely a tenth of what Subversion required.)

Alternatives
While I could have run git-svnimport directly against the remote repository, this would have taken about 32 hours. Even using svm or SVN::Mirror in place of rsync would have required around 10 hours.

If you can use rsync or another tool to make a local copy of the repository, the import will run much more quickly.

Tuesday, January 09, 2007

Fixing Synergy Shift Behavior

When running the Synergy client on CentOS with a Window machine operating as the server, I had problems with the behavior of the [Shift] key. There was a delay from when [Shift] was pressed until it took effect and often the left [Shift] key would not take effect at all.

This was corrected by downloading, building and installing the latest source RPM from the Synergy site. You may have as much luck with the binary RPM. I just didn't try it.

Update: Fixed typo

Tuesday, December 19, 2006

BOINC and Tagging?

Following yesterday's conference, I couldn't help but keep coming back to the idea of tagging. Yes, I really do mean the popular concept of tagging used on so many social web sites like del.icio.us and MySpace.

Why tagging? It's popular and it could provide a possible answer to some of the problems discussed yesterday.

Finding Projects
If projects had various tags, a user could choose to subscribe to or ignore projects based on different tags they were labelled with. These tags could used by the account managers and other sites. Then as projects appeared and disappeared they could automatically be added to your client, if you chose to do so.

User Visibility
One of the questions yesterday had to do with retaining people and them being able to see where they sat in the BOINC world. Were they advancing or falling in the listings? Did their contributions matter?

What if users tagged themselves? Say I tagged myself with: Male, USA, Single-Computer, Linux, Missouri, MoBot. Then I could see how I placed within each of those tags. I may not show up on the overall CPDN list, but I might under Single-Computer+Linux+Missouri for example.

Well, good or bad, those were some of my thoughts.

BOINC Conference Summary

Here is a "brief" summary of the BOINC conference that was held online, December 18th. It took place over Skype with people from all over the world. At times the connections were a little week (at least on my end), but it was a very worthwhile couple of hours.

This comes from my quickly scribbled notes. Nothing is a direct quote (unless my hand and memory got lucky) and I am sure there are many errors. I may have even missed a question or two.

If you want the definitive source, get the mp3. It should be available shortly from the BOINC UK site.

The questions mostly broke down into two basic categories. Those about BOINC in general and those specifically related to SETI@home.

General Question
If you had one wish what would it be?

BOINC Questions
With all of the optimized BOINC clients available why do we still need the "vanilla" client?
How can we resolve issues with large workunits?
Have you considered subteams or a way to give users more visibility?
What scalability issues are there as the number of projects continues to grow?
What is the status of using GPUs for computation?
Question about clients and credits.
What do you see as the relationship between BOINC and the media?
What about sandboxing?

SETI@home Questions
What is the status of the Southern Hemisphere data recorder?
What does a data recorder cost?
What it the likelihood of Arecibo shutting down?
What is the status of optical SETI work?
What percentage of stars have been listened to?
What about picking up leakage of TV, radio, and other broadcasts?
What is the projected rollout date for astropulse?
What about VLA data?

Q. With all of the optimized BOINC clients available why do we still need the "vanilla" client?

A. The optimized client was a kludge for the older version of SETI so that users would get the "correct" amount of credit. Now that SETI has moved to flops based computation there is no need for the optimized client.

Now, if you are actually talking about optimized applications (projects) as opposed to the BOINC client software that coordinates fetching workunits and running the projects on your local machine, they help. But, the issue is that there are two many distinct architectures and sub-architectures to properly test all of them.

Q. Due to the size of the Climate Prediction work units, many users simply crash and are never heard from again. How can we resolve this? Perhaps with popups and other messages? --- Mo (Maureen?)

A. We support trickle messages that can update the user as they continue to work on very large work units. Climate Prediction could do a bit of work to provide screensaver updates through using these trickle messages.

While some would like popup messages that let them know when something goes awry or needs attention, others would see this as intrusive. --- Dr. David Anderson

Q. What is the status of the Southern Hemisphere data recorder for SETI at the Parkes Observatory? --- Mark (Ohio, USA)

A. With the threatened shut down of the Arecibo telescope in 5 years we have to focus there and consider the possible transfer of equipment after that. --- Dr. Eric Korpela

Q2. What does a recorder cost? --- Mike

A2. It depends upon the donations of equipment, but even with the donations received, the original recorder cost around $300,000 (US). --- Eric

Q3. What is the likelihood of Arecibo shutting down? --- Mike

A3. The problem is that there are no congressmen with voting power for Arecibo. It is in Puerto Rico and this is really a political financial decision. --- Eric

Q. What about Optical SETI work? --- Tiare (Chile)

A. The optical work we are doing is relatively primitive compared to the radio work. It would be highly unlikely for each of our three sensors to receive a flash, although it has been guessed that a laser pulse directed at us from extraterrestrial sources could give a pulse at all three wavelengths. --- Eric

Q. If you had one wish what would it be?

A. Something indistinct about Microsoft. --- David
Better hardware --- ???
A verifiable contact --- Eric
Accurate benchmarking --- David

Q. Have you considered subgroups within teams so that users would have a better chance of seeing themselves showing up on a top 10?

A. We thought that if the servers generated sufficiently refined lists based upon data available to them that this would happen automatically. So far, it has not.

Q. What percentage of stars have been listened to? --- John (USA)

A. Arecibo has had pretty good coverage, but it only sees about 25% of the sky. Multi-beam is 30%(?) more sensitive and allows us to listen to sources that are 5 times more distant. We have also done some targeting with high bandwidth, but it is a tradeoff, bandwidth vs. coverage. We can listen to a lot on a few frequencies, or listen to a few things on a lot of frequencies.

When you look at all three, bandwidth, distances and sky coverage, we have had relatively little overall coverage. --- Eric

Q. What about picking up leakage of TV, radio, and other broadcasts?

A. Omnidirectional signals like those do not go very far. Even with a directed beam we can only reach about 50(?) lightyears with broadcasts of our own, say from Arecibo.

Q. It seems the number of projects is growing rapidly. What are the limitations of BOINC as this continues? --- Maureen

A. Since the BOINC project does not supply centralized servers,there are really no limits with respect to that. It is more a problem in terms of project discovery. BOINC account managers are one solution to that problem. BOINC mutual funds may be another.

BOINC mutual funds would be where an individual could say they are interested in computing for cancer research, but they don't want to have to keep track of cancer research projects as they come and go, so they subscribe to a "cancer research mutual fund" that would keep track of these things for them. Essentially this would be using intermediaries like the American Cancer Society to keep track of these things. --- David

Q. ATI has expressed interest in working with BOINC on producing GPU accelerated interfaces and Folding@Home has had success with a similar initiative. What is the status or progress in this area? --- Ron (Missouri, USA)

A. We tried doing some work with the NVIDIA GPUs but the architecture was not really conducive to integrating with them. It is our understanding that ATI may have a more accessible architecture, but so far no projects have bit on the offer to work with them.

We are looking at using the Cell Processor in the PlayStation® 3. It may be much better. It is several times faster than a GPU and about 50x faster than a standard processor. It also give much better computational power per Watt of electricity making it more environmentally friendly. --- David

I have not given up on GPUs. If there are any GPU programmers out there that are interesting in helping out on SETI@home, I would be very interested to hear from them. --- Eric

Q. Indistinct --- Tiare (Chile)

A. We have created a new simple GUI in cooperation with the Community Grid. It will make the BOINC client skinnable. We are also looking at a unified credit system that would allow us to grant credit for donated storage space as well as for donated CPU time. We would like to retain a single credit scoring system. But, benchmarking this is difficult. --- David

Q. What is the projected rollout date for AstroPulse? --- Derek (Guam)

AstroPulse is a long promised set of work units for SETI@home. According to a previous interview there had been a grad. student that was supposed to be working on it, but he found it more interesting to work on other things, so the effort that was put forth never came to fruition.

A. We have a new grad. student working on it. Hopefully it will be in beta next month or so. That would be nice, since he will have some time off of class then. --- Eric

Q. When the BBC did its documentary on Climate Prediction.Net it greatly enhanced the participation in the project. What do you see as the relationship between BOINC and the media? --- Maureen

A. Media coverage helps greatly. But many of those participants did not stick around long, for reasons we've already discussed. --- David

Q. Indistinct question about Sandboxing? --- Ian

A. We are using user-account based sandboxing and we recommend that the application runs as a limited user. We have this setup on the Mac already. It works with a little help on Linux as well. The problem (as usual) is Windows. Although we expect to have a Windows version of sandboxing available in about 2-3 month. --- David

Q. Something about VLA usage? --- Derek (Guam)

A. None currently. Allen(?) telescope is expanding to 45 dishes. [Garbled] needs more dishes, less than 45 does not provide what we need. Originally [garbled] was supposed to have around 200 telescopes, but due to cutbacks this has not happened. --- Eric

Well that's all I have from my notes, and a little memory thrown in.

I'd like to thank Mike and all those that arranged the conference and David and Eric for taking time out to answer our questions.

Monday, December 18, 2006

Visual SourceSafe 2005 on Linux

Getting Visual SourceSafe 2005 working on Linux was a little tricky. It required a lot of experimentation, and I did not keep exact notes. So, this is just a rough outline of how I got it working.

Start by getting the latest version of wine from git. When you configure it, install any missing dependencies and reconfigure. Then, build and install it.

Install ies4linux. Only Internet Explorer 6 is required. I used a base directory of $HOME/.wine when I ran the installation script:
./ies4linux --basedir $HOME/.wine
Then, I moved all of the contents of $HOME/.wine/ie6 into $HOME/.wine and deleted the ie6 subdirectory. If you plan on using the desktop icon, you will need to edit it to match the modified path.

Run winecfg and change the Windows Version setting to Windows 2000.

The Microsoft .NET Framework Version 2.0 must also be installed. However, the installation procedure does not appear to work under wine.

So, run cabextract on the download and then run it on the netfx.msi. This extracts a lot of files with really long names. Look at the file whose name contains _INF_. In my case this was
FL_NETFXUSA_INF_92947_____X86.3643236F_FC70_11D3_A536_0090278A1BB8.
It contains a list of files and target directories.

Some of the target directories will exist in .wine/drive_c, others will need to be created.

Then, you have to find the files listed for installation into those directories. They will not appear under in the extraction directory with the names shown in the info file. Rather, they will appear with a name like the FL_NETFXUSA_INF_... file shown above.

Using it as an example, the short name would be netfxusa.inf. Simply copy those files into the appropriate target directory under the shorter name.

When I did this, at least one file was specified to be copied into two different directories.

Visual SourceSafe requires some fonts that are not normally installed for wine. I suspect it really needs Tahoma, I didn't try to narrow it down, I just copied my Windows\Fonts directory from my XP partition into the .wine/drive_c/windows/fonts subdirectory.

Copy the Program Files subdirectory from the VSS installation CD into the wine/drive_c directory. Then, run
wine $HOME/.wine/drive_c/Program\ Files/Microsoft\ Visual\ SourceSafe/ssexp.exe
If everything goes well, Visual SourceSafe should display with some minor toolbar and menu issues. If you don't see a menu, pass your mouse over where it should be and the top-level items will appear.

You might want to copy that command into a script file or create a launcher for it instead of having to type that command every time.