Listing SOLR Collections, Java

Every now and then I need to get the list of collections in a SOLR cloud instance from a Java program. The docs seemed opaque on how to do it, so:

CollectionAdminResponse response = new    CollectionAdminRequest.List().process(server);
ArrayList cols =      (ArrayList)response.getResponse().get("collections");

Java/Spring Caching

File it under "another thing I need to do a lot"... Some kind of caching of java objects.

Turns out there is a neat cache library from google called Guava that's very easy to inject into Spring and easy to configure and access.

The Spring beans look like this:


<bean id="cacheBuilderSpec"
  class="com.google.common.cache.CacheBuilderSpec"
  factory-method="parse"
  c:cacheBuilderSpecification="${cacheSpecification}" />

<bean id="cacheBuilderFromSpec"
  class="com.google.common.cache.CacheBuilder"
  factory-method="from"
  c:spec-ref="cacheBuilderSpec" />

<bean id="cache"
  factory-bean="cacheBuilderFromSpec"
  factory-method="build" />

And where the injected $cacheSpecification looks like this:

cacheSpecification=expireAfterAccess=12h,maximumSize=100

Using it in my code then becomes a simple matter of injecting the 'cache' bean into whatever class I'm using and then accessing it with these kinds of code:

cache.getIfPresent(object)
cache.invalidate(object)

Installing Oracle Java in Ubuntu, Something I seem to do a lot!

I create Virtual Box Virtual Machines for just about every project, so I end up installing Java on them a lot. I like the Oracle JDK, but for licensing reasons, that install is not available as part of the "normal" apt repository.

I installed Java via a PPA (http://askubuntu.com/questions/4983/what-are-ppas-and-how-do-i-use-them) by webupd8team (http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html). I ran these commands to add the PPA to the apt-get repo and update that repo, install Java7, and set up the various java 7 environment variables.

add-apt-repository ppa:webupd8team/java

apt-get update
apt-get install oracle-java7-installer

apt-get install oracle-java7-set-default




This installs the java 7 version. Same commands work with 8 by changing the version number.

Creating a Java, Marklogic development stack on an Ubuntu 13.10 VirtualBox VM

I wrote about building a Java-based development stack using Ubuntu on a VirtualBox VM some time back (http://www.tractare.blogspot.com/2011/10/ubuntuvirtualboxjava-team-development.html, and http://www.tractare.blogspot.com/2011/10/ubuntuvirtualboxjava-team-development_04.html). But that was a couple of years ago and the technology has changed, so this is a rewrite. Also, that version didn't include Marklogic, which is a part of the stack for this particular application. So here's the stack I'm building out and the steps I took to get there.

First off, I'm building this in Virtual Box  4.3.8, installed on a Dell XPS laptop quad-code I7, 16GB RAM and dual 750GB HDD running Windows 7. Second, some of these kits I installed using PPAs, so if you have security concerns about PPAs, you'll need to find other ways for those tools.

The development stack I'm installing is this:

• Ubuntu 64bit 13.10 with VB Guest Editions
• DropBox
• Oracle Java 7
• JEdit
• Git 
• Maven
• Tomcat 7
• Chrome Browser
• Eclipse Kepler
• Marklogic 7

Installation Process:
A. Create the Virtual Machine...

1. Downloaded the Ubuntu 13.10 64bit Desktop ISO  image from http://www.ubuntu.com/download/desktop
2. Created a new VirtualBox VM targeting a linux/Ubuntu 64 bit OS with 10GB Ram and an 80GB Drive.
3. Started that new VM and pointed it at the ISO image to install. (and chose to install rather than to try Ubuntu).
4. After answering all the usual questions (where am I), restarted the VM and mounted the VB Guest Additions. The ran them via the autorun. Installing Guest Editions is a critical part of making a useful VM in VirtualBox. Don't leave home without it.
5. Shut down the VM and opened the VirtualBox Manager Settings for my new VM; Changed the following:
6. General->Advanced:
   Shared Clipboard=bidirectional
   Drag'n'Drop=Bidirectional
7. System->Processor:
   4 CPUs
8. Display->Video: 128MB
9. OK to save all that, then start the VM again.

B. General VM Stuff, and Dropbox...

1. The default gnome desktop has a bunch of stuff I don't use (word processing and so on),  so I removed and uninstalled them using the Ubuntu Software Center.
2. I like to be able to do things in root, so I changed the root password. Saves having to sudo all the time!
   sudo passwd root
3. Next, opened a terminal (and pinned it to the launcher) and started in on the stack. and I switched to root user (su root).
4. Dropbox: I run Dropbox on the Windows machine and I don't see the point of running it in the guest Ubuntu VM as well. But I like having those folders and files available. So I create a shared folder and mount that in the Ubuntu VM.
   
   1. in Devices->Shared Folders Settings, create a new "Machine Folder" that points at the Dropbox folder on the Host.
   2. Create a directory of the same name in Ubuntu (Dropbox, in my case).
   3. Mount that:
      sudo mount -t vboxsf Dropbox /home/cup/Dropbox

C. Java, JEdit, Git, Maven...

1. Java: If this is a new, clean Ubuntu install then there is no pre-installed Java. If you are trying to upgrade, you should check for and remove any extant Java installation.
   
   I installed Java via a PPA (http://askubuntu.com/questions/4983/what-are-ppas-and-how-do-i-use-them) by webupd8team (http://www.webupd8.org/2012/01/install-oracle-java-jdk-7-in-ubuntu-via.html). I ran these commands to add the PPA to the apt-get repo and update that repo, install Java7, and set up the various java 7 environment variables.
   

   
   add-apt-repository ppa:webupd8team/java
   apt-get update
   apt-get install oracle-java7-installerapt-get install oracle-java7-set-default
   
   

2. JEdit: Installed this via the software center. This installed version 5.which is good enough for my purposes. 
3. Git: simple - sudo apt-get install git
4. Maven -- also simple sudo apt-get install maven
5. Tomcat7 -- Installed directly from the Ubuntu Software Center.

D. Chrome Browser...

1. Chrome Browser: Via PPA:
   
   wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | sudo apt-key add -
   sh -c 'echo "deb http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google-chrome.list'
   apt-get update
   apt-get install google-chrome-stable

E. Eclipse...

1. Eclipse: A bit of a pain in the neck, I followed this guide: http://blog.thismagpie.com/2012/07/installing-eclipse-juno-42-on-lubuntu.html. Downloaded the Eclipse Linux 64 tar.gz distribution and saved that file. Then ran these commands as root:
   

   
   tar -zxvf eclipse-...
   sudo cp -r eclipse /opt
   cd /usr/local/bin
   cp -r /opt/eclipse/eclipse eclipse
   
   After that, I ran eclipse from the command line and then pinned it to the launcher.

2. One issue I had with Eclipse was that the command menus didn't work. I had to edit the eclipse.desktop file in ~/.local/share/applications to get it to work by commenting out the Exec and adding this one:
   
   Exec=env UBUNTU_MENUPROXY= /usr/local/bin/eclipse
   

F. Marklogic...

1. Downloaded Marklogic7 for 64Bit RedHat Linix and ran these commands
   apt-get install alien
   alien --to-deb --verbose MarkLogic-4.1-6.x86_64.rpm
   dpkg -i marklogic_4.1-6_amd64.deb
   /etc/init.d/MarkLogic start
   
   The one issue I had with the ML install is that the Marklogic script in /etc/init.d runs under /bin/sh and Ubuntu's default shell is bash. So I changed the first line of that script to #!/bin/bash

And that's it. I now have a reasonable development stack to create Web based Java code using Marklogic.

Ubuntu/VirtualBox/Java Team Development Stack - Part II

Part II - Install the Development Stack.
Continued from Part I

Having a virtual machine with Ubuntu installed, we now need the developer tools in our stack.

My target developer's stack has:

1. Java JDK
2. Eclipse
3. Ant
4. jEdit
5. Firefox (or Chrome)
6. Tomcat
7. MySQL 
8. MySQL Client
9. DropBox

It's worth installing Ubuntu Tweak. At the least, this gives us the window icons on the upper-right corner. Open a browser and go to http://ubuntu-tweak.com/. Install that.From within that, I also found the Chrome browser install and did that.

The approach Ubuntu uses for typical software installation is through the "Ubuntu Software Center" which is invoked from the bottom of the Applications menu. Easiest is to use the search box to find the software we want and then choose the version (if there are options) to install. I simply searched for and installed each package. Some notes:

1. Java JDK: I chose the "OpenJDK Development Kit".
2. Tomcat is installed on port 8080. Something else must be running on port 80 so I wasn't able to change the tomcat port.
3. Tomcat installs as a service. As such, it can be stopped and started: service tomcat6 stop|start|restart
4. I added a manager role and user ID/Password so I can access the tomcat manager and use the Eclipse/Ant deploy tasks.
5. I created a new folder in home: /home/projects and made it read/write/execute accessible to the users group. 
6. When I installed dropbox (by going to the www.dropbox.com website and installing from there), I pointed Dropbox at the projects directory so that those files would be under /projects.

Ubuntu/VirtualBox/Java Team Development Stack - Part I

Part I - Setting up a Virtual Machine and Ubuntu

A controlled java web development stack for a distributed development team.  I did this a few times before but never documented it, and (of course) versions of the tools have all moved on. But I have to resurrect it, so this time I think I'll write it up. No where near as easy as Dave Winer's EC2 for Poets (http://poets.scripting.com/), but he's a much better writer than am I!

So, the idea is this: several developers; some co-located and others not; working on a web project being developed in Java. This project is also using MySQL for an underlying database. Code is managed in a remote CVS repository. What I want to create is a controlled environment in which each developer has the same set of tools and test data against which to work. The answer: a virtual machine with all the tools installed and configured so that I can simply deliver a copy of the VM disk image to each developer. I've played with this before in VMWare and some other virtual machine kits that I don't recall, but most recently in Oracle's VirtualBox. So I'm repeating that experience here. It  bears noting that I am doing this stack creation on a Windows Vista 32-bit laptop (no flaming please). But I believe the overall approach here would work on pretty much any host machine. Also of note: since I don't want to deal with licenses (and fees) for virtual machines that I send out to developers, I am using Linux Ubuntu as the virtual client. This being a java development project, what we develop is (or certainly should be) portable across different operating systems, but the production target for this is also Linux, though not Ubuntu.

My target developer's stack has:

1. Java JDK
2. Eclipse
3. Ant
4. jEdit
5. Firefox (or Chrome)
6. Tomcat
7. MySQL
8. A VPN connection for CVS

In addition, there needs to be a way to share files from the host machine to the virtual machine. Here are the rough steps I followed:

A. Create an ISO Image of Ubuntu:

1. Download Ubuntu (http://www.ubuntu.com/download/ubuntu/download). 
2. The ISO image will need to be available to VirtualBox for installation, either on a CD or a USB drive.  I use a USB drive. Just copy the ISO image to that.
   

B. Set up a Virtual Machine with Ubuntu:

1. VirtualBox: https://www.virtualbox.org/wiki/Downloads. This is  simple enough to install - just follow the steps and accept the defaults. Once it is installed, start it and create a new virtual machine.
2. New Virtual Machine: I chose Linux/Ubuntu and then set memory to 1Gb.
3. New "Hard Disk": I set it to dynamically expanding.
4. At this point, I'm done creating the virtual machine - I need to install Ubuntu. By clicking "start" on the newly created machine, I get the "first run" wizard. From there, point to the USB drive from A. above.
5. Follow the setup for Ubuntu. 

C. Create a root Password.

By default, Ubuntu is installed without a root login. I'm guessing this is for security reasons. But these developer VMs are not really subject to security issues and having the ability to run as root makes a lot of things easier. So I add a root password, enabling me to su and run as root when I need.

1. sudo passwd root

First, you'll have to enter your password. Then the password you want for root.


D. Add my ID to the "users" group.
I found that my user ID was in its own group. I added "users" as my group so that I can access common folders more readily.

E. Install Guest Additions

This is a big/important key step. It makes many of the hardware and device features of the host computer pass through to the virtual machine. Like the screen size/resolution. And others.

1. There is a virtual machine menu bar with a "devices" menu. This has a "Install Guest Additions..." item. Clicking this will *not* actually install the guest additions; it only mounts the guest additions as a virtual drive. So I did this, but then still had to run the actual install.
2. After the mount, open a terminal window and cd to the mount for the additions. These seem to be under /media.
3. Run the autorun.sh (sh ./autorun.sh). I switch to root before doing this (su).
4. Restart the server.

F. Ensure Networking Works.
Since we need to access CVS over a VPN, we need the VPN access to work. For our purposes, each developer has VPN access from the host machine (my laptop). So I want this to "pass through" to the virtual machine. I'm not a networking guy, so I don't know much about this, but it seems that by setting the network adapter in the virtual machine to "NAT" (which seems to be the default anyway), I go the pass-through I wanted. Perhaps someone more knowledgeable than me can elucidate? Anyway, I simply tested this by connecting to the VPN and pinging a machine on our internal network.

G. Set up Sharing.
We'll want to be able to share files from the host file system to the virtual machine file system. This means setting up a Shared folder.

1. On the host machine, create a folder (I called it "Sharing" and located it in the root).
2. In the Virtual Machine, create a permanent share - from the "Devices" menu, select "Shared Folders..." and create a new share. I created a "permanent", "Auto Mounted" share and named it Sharing. 
3. After restarting the VM, this share is now found in /media/sf_Sharing.
4. I had to change my user ID and add "vboxsf" as a group to be able to access this.

Continued: Part II - Install the Development Stack.