Tuesday, May 18, 2010

Managing Oracle Tablespace (ORA-01653 error)

This procedure helped me a bit to manage an Oracle DB's tablespace. I was getting ORA-01653: unable to extend table SYSTEM... errors.
Maybe the following procedure is useful for others as well:
  1. In order to manage/see the current usage, here's a good script posted by 'bipul' on http://forums.oracle.com/forums/thread.jspa?messageID=3590569
  2. To check which data files are used:

    select name from v$datafile

  3. To increase that setting:
alter database datafile 'C:\ORACLEXE\ORADATA\XE\SYSTEM.DBF' autoextend on next 100m maxsize 2000m;

Wednesday, February 24, 2010

Quants

An interesting documentation about quants, including Emanuel Derman,
starts in Dutch first, but in English later.

Monday, February 15, 2010

Java: Sorted List

Sometimes you might have the need for a Java list, that is sorted with every element insertion. If you use a sorted TreeSet, you are stuck with iterators, no random access for elements. If you use an ArrayList, you could call Collections.sort() after every add, but that's not necessarily efficient. Collection.sort() will have to check the order for every element. If your list is long, this might take time.
But here's the trick: if you know your list was already sorted before your insertion, there's no need to re-sort everything again. You can find the proper insertion point with a quick binary search, and insert the new element at the proper position. VoilĂ , your list is always sorted.
Surprisingly simple to do this in Java:



/** A LinkedList that efficiently sorts itself with every add.
 * 
 * @author apodehl
 *
 */
public class SortedList< T extends Comparable> extends LinkedList
{
  /** Adds this element to the list at the proper sorting position.
   * If the element already exists, don't do anything.
   */
  @Override
  public boolean add(T e)
  {
    if( size()==0 ) {
      
      return super.add(e);
    }
    else {
      
      // find insertion index
      int idx = -Collections.binarySearch(this, e)-1;
      
      if( idx < 0 ) {
        return true; // already added
      }
      
      // add at this position
      super.add(idx, e);
      return true;
    }
  }
}



Java BiMap, BidiMap

After doing some research and trying to find just the right Java Bi-Map, I couldn't find any, so I had to roll my own :-( To my surprise, implementation of a Bi-Maps is not straight-forward, there's not such thing as the 'one Bi-Map functionality'.
The interesting case is this one: say there are mappings A->1 and B->2. What happens once you insert C->2 ?
Will you remove B->2 to ensure uniqueness of keys and values ? (resulting in A->1 and C->2)
or not, resulting in A->1, B->2 and C->2. Please note that in the latter case, there's no well-defined lookup for value "2". Is it B or is it C ?
So what I needed was uniqueness of both keys and values, in all cases, using generics. Here it goes:


import java.util.Collection;
import java.util.LinkedHashMap;
import java.util.Map;
import java.util.Set;

/**
 * One possible implementation of a Bi-directional Map.
 * 
 * Note: there are several variations of BiMap, depending on the behaviour of put(k,v)... Is it allowed to have more
 * than one k for one value ?? Modify
 * 
 * @author apodehl
 * 
 * @param 
 * @param 
 */
public class BiMap implements Map
{
  Map keyVal; // maps key->value

  Map valKey; // maps value->key

  public BiMap()
  {
    keyVal = new LinkedHashMap();
    valKey = new LinkedHashMap();
  }

  @Override
  public V put(K key, V val)
  {
    // --- this implementation allows one-to-one ONLY !! ---
    if (valKey.containsKey(val)) {
      keyVal.remove(valKey.get(val));
    }
    if (keyVal.containsKey(key)) {
      valKey.remove(keyVal.get(key));
    }
    // --- remove above if that's not what you want ---

    // 
    valKey.put(val, key);
    return keyVal.put(key, val);
  }

  @Override
  public void putAll(Map< ? extends K, ? extends V> m)
  {
    for (Entry< ? extends K, ? extends V> e : m.entrySet()) {
      put(e.getKey(), e.getValue());
    }
  }

  /**
   * Type-safe get.
   * 
   * @param key
   * @return
   */
  public K getKey(V val)
  {
    return valKey.get(val);
  }

  /**
   * Type-safe get.
   * 
   * @param key
   * @return
   */
  public V getVal(K key)
  {
    return keyVal.get(key);
  }

  @Override
  // unfortunately Map interface isn't type-safe
  public V get(Object key)
  {
    return getVal((K) key);
  }

  /**
   * Type-safe contains.
   * 
   * @param key
   * @return
   */
  public boolean contains(K key)
  {
    if (keyVal == null) return false;
    return keyVal.containsKey(key);
  }

  /**
   * Type-safe containsValue.
   * 
   * @param key
   * @return
   */
  public boolean containsVal(V val)
  {
    if (valKey == null) return false;
    return valKey.containsKey(val);
  }

  // unfortunately Map interface isn't type safe here ..
  @Override
  public boolean containsKey(Object key)
  {
    return contains((K) key);
  }

  // unfortunately Map interface isn't type safe here ..
  @Override
  public boolean containsValue(Object value)
  {
    return containsVal((V) value);
  }

  @Override
  public Set< java.util.Map.Entry> entrySet()
  {
    return keyVal.entrySet();
  }

  @Override
  public boolean isEmpty()
  {
    return keyVal.isEmpty();
  }

  @Override
  public int size()
  {
    return keyVal.size();
  }

  @Override
  public Collection values()
  {
    return keyVal.values();
  }

  public V removeByKey(K key)
  {
    V val = keyVal.remove(key);
    valKey.remove(val);
    return val;
  }

  public K removeByVal(V val)
  {
    K key = valKey.remove(val);
    keyVal.remove(key);
    return key;
  }

  // unfortunately Map interface isn't type-safe
  @Override
  public V remove(Object key)
  {
    return removeByKey((K) key);
  }

  @Override
  public Set keySet()
  {
    return keyVal.keySet();
  }

  @Override
  public void clear()
  {
    keyVal.clear();
    valKey.clear();
  }

  public String toString()
  {
    String s = "BiMap:\n";
    s += "  key->value: " + keyVal.toString();
    s += "\n";
    s += "  value->key: " + valKey.toString();
    return s;
  }

  public static void main(String[] args)
  {

    BiMap biMap = new BiMap();
    biMap.put("A", 1);
    biMap.put("B", 2);
    System.out.println(biMap); // {A=1, B=2}

    biMap.put("C", 2);
    System.out.println(biMap); // {A=1, C=2}

    biMap.removeByVal(1);
    System.out.println(biMap); // {C=2}
  }
}

Wednesday, December 23, 2009

Redundant home internet with a Surf Sticks and Internet Connection Sharing

If you have a not-so-reliable ISP and an UMTS surf stick you can configure your home internet to be more redundant by setting up the surf stick as secondary gateway and DNS server for all your machines on a local network.
Takes a while to set up, but its worth it :-)
Assumption: you already have a primary internet gateway (e.g. DSL/Cable) on 192.168.0.1.
Internet host setup (sometimes connected directly through a surf stick):
  1. Setup your dial-up connection to use 'Internet Connection Sharing' (ICS) as described, for example here: http://support.microsoft.com/kb/306126
  2. Make sure that for Internet Connection Sharing 'Settings' you have enabled DNS to be provided by this new Internet host. Possibly DHCP, but not for now.
  3. Also make sure that the LAN network interface (on a laptop, that's usually 'Local Area Connection') now uses a static IP. Set it the way you would like your new internet gateway to appear on your home network. For example in my case, I already have a default gateway on192.168.0.1, so the new Internet-Host, should still use this primary gateway while it is not connected through the surf stick. The new internet host's ip is 192.168.0.100
Ethernet adapter Local Area Connection:
Connection-specific DNS Suffix . :
Description . . . . . . . . . . . : Intel(R) 82567LM Gigabit Network Connection
Dhcp Enabled. . . . . . . . . . . : No
IP Address. . . . . . . . . . . . : 192.168.0.100
Subnet Mask . . . . . . . . . . . : 255.255.255.0
Default Gateway . . . . . . . . . : 192.168.0.1
DNS Servers . . . . . . . . . . . : 192.168.0.1
So the Internet Host (my laptop with a surf stick) will use IP and DNS from 192.168.0.1 if it currently isn't connected directly.
Client Machine Setup:
The client machines on the network 192.168.0.x must be configured to use both, the primary gateway at 192.168.0.1 and, if not available, the secondary gateway at 192.168.0.100.
On each client machine, go to 'Local Area Connection' -> Properties -> TCP/IP ->Properties and specify an "Alternate DNS Server" of 192.168.0.100 and click 'Advanced' to add a new Default Gateway. Click 'Add' and set the 'TCP/IP Gateway Address' to 192.168.0.100.
Now each client uses 192.168.0.1, while your DSL/Cable internet is available, and 192.168.0.100 otherwise.
The whole configuration with DHCP is similar, you just need to prepolute the DHCP answers with about the same settings (Default gateways and DNS: 192.168.0.1 and 192.168.0.100) .

Friday, October 30, 2009

MAC/IP lookup

I was trying to figure out more about a MAC address and found an interesting lookup mechanism:
To be concise, all lookups on one page:

Thursday, October 29, 2009

Automatic Restart Script for a Java service

This blog tries to describe a pattern on how to write a pragmatic Unix start and stop script for an automatically restarting Java service.
In many cases, you might want to write a Java service that should be up and running 24x7. Now in theory the garbage collector should deal with everything, and if programmed correctly, the process should never crash. But in praxis, things do happen. For example a web service could encounter user requests where memory use is much bigger than you ever expected.
What is a good maximum heap size anyway ?
A pragmatic approach is just to face the fact that your JRE could run out of memory and deal with it. There are sophisticated monitoring solutions out there to automatically restart processes (e.g. nagios/ganglia), but a poor-man's solution is to automatically restart the JRE from the Unix start script.
Please note, you don't want to restart in every case. A bad command line option should just stop the process and not run into the restar loop. Also, there must be a clean way to manually stop it.
Under these constraints, the best solution I could find is to create a temporary file from the Java code at exactly the point 'of no return'. If the JRE stops before this point, no restart happens. If the JRE stops after this point, automatic restart will kick in.
... parse command line options ...
 
     // register shutdown hook      Runtime.getRuntime().addShutdownHook(new ShutdownHook(...));
     // register uncaught exception handler
    Thread.currentThread().setDefaultUncaughtExceptionHandler( new UncaughtExceptionHandler() {
      public void uncaughtException(Thread t, Throwable e)
      {
        e.printStackTrace();
        if( e.getClass()==java.lang.OutOfMemoryError.class ) {
      
          System.err.println("FATAL: shutting down because of java.lang.OutOfMemoryError ...");
          System.exit(7);
        }
      }} );

    File restart = new File("webservice.restart"); // tell the shell to restart me
    restart.createNewFile();

... start your service ...
The Unix start script below will restart the JRE if, and only if, the temporary file (webservice.restart) exists. This could go like this:
#!/bin/sh

#try to start service once
${JAVA_HOME}/bin/java -DREPLAYWEB ${JOPT} com.codestreet.replay.jms.shell.web.ReplayWeb $*

# restart again (until webservice.restart file was removed)
while [ -f webservice.restart ]; do
echo "### RESTART ###"
/bin/rm -f webservice.restart # let Java decide if we really want to restart
${JAVA_HOME}/bin/java -DREPLAYWEB ${JOPT} com.codestreet.replay.jms.shell.web.ReplayWeb $*
done

So Java decides if the service should be restarted and the shell actually performs the restart.
An alternative would have been to simply use return codes from System.exit() ?
But then the question would be: what's the return code with a not yet known exception ?
If someone else uses kill-9 on the jre, a shutdown hook wouldn't be invoked. And to manually stop the restarting you would have to kill the start script as well as the JRE.
With this file-based approach, the stop script is pretty simple: remove the temporary file and kill the JRE. Please note that finding the proper Java process is not as simple as it seems since the classpath is usually very long and 'ps -f' potentially won't show the classname anymore because the line gets too long. On Linux you can use the --col option to see a longer output, but that doesn't work on Solaris :-(
So a little trick around this is to use a dummy JRE property, -DREPLAYWEB in this case. This mock up property has no meaning except that it will show up in ps before the classpath and you can make it unique enough to identify only this instance of JRE.
The stop script would then perform these steps:
  1. get the 'ps line' that contains the dummy JRE property (REPLAYWEB)
  2. get the task id of that process (awk is good enough)
  3. remove the temporary restart file so the start script won't restart automatically
  4. kill the process
#!/bin/sh -f

psline=`/bin/ps -aef | /bin/grep "REPLAYWEB" | /bin/grep -v grep`
echo $psline
pid=`echo $psline | /bin/awk '{ print $2}'`
if [ $pid ]
then
 /bin/rm -f webservice.restart
 echo "stop_webservice: killing Web Service with pid=$pid"
 kill $pid
else
 echo "stop_webservice: Web Service was not running"
fi