Aggregate bitwise OR in SQLite

The following SQL snippet applies bitwise OR on all the values in numbers table's value column. This example supports up to 8 bits; you can easily extend it to support more.

CREATE TEMP TABLE numbers(value INTEGER);
INSERT INTO numbers VALUES(2);
INSERT INTO numbers VALUES(3);
INSERT INTO numbers VALUES(15);

select 
((sum(value&1)>0) << 0) + 
((sum(value&2)>0) << 1) +
((sum(value&4)>0) << 2) +
((sum(value&8)>0) << 3) +
((sum(value&16)>0) << 4) +
((sum(value&32)>0) << 5) +
((sum(value&64)>0) << 6) +
((sum(value&128)>0) << 7) 
from numbers;

Choose conferences with nice locations (Being Grad Students)

One of the nicest being a grad student is you get to travel, often "for free", to conference held at great places (e.g., Sydney, Paris, Hong Kong, etc.).

Picking such a conference could motivate you to work hard for a paper. :-) So choose your conference wisely. After all. If you have some good work that you can submit to similar conferences that have comparable reputation/prestige, which conference would you send it to? The nicer one of course.

Just remember to check with you advisor that it's OK with him/her though.

Cut your own hair (Being Grad Students)

When I was little, my father cut my hair. Then I came to the states for grad school -- no one cut my hair anymore!

Going to a hair salon in the US can be so expensive. The costs add up really quickly. So I thought about doing it myself; I remembered my father saying "it's a piece of cake" cutting my hair. How bad could it be if I did it myself?

Very bad, when I first started, a few years back. I bought a cheap Wahl hair clipper (like this one),  and I hated it, even though it's a 5 star product.
1) It has a long cord, which often get in the way
2) It's really awkward holding it to cut the side or the back of my head -- just try using your right hand to scratch your left ear from behind your head, and you'll know what I mean.
3) It's a big hassle having to keep changing the "length guides" for different hair lengths.

Apparently, you need something different when you cut your own hair. I found this Philips hair clipper recently, which solved all the above problems!!
1) It's cordless.
2) It has a rotating head, so you can hold it sideway when you cut your hair on the side or the back of your head
3) It has a "zoom ring" that lets you change the cutting length from 30mm (about 1.18 in) down to super close shave. No need to fiddle with the millions of "length guides"! (Well, technically, there are still two guides, one for the longer lengths, one for shorter lengths.)

I love this clipper. Great for a quick fix.

For those of you (men) who haven't cut your own hair before. It's very easy (assuming you use this Philips clipper, and you just want to style your hair a little). Here's how:

1a) Pick the longest length you want your hair to be (e.g., 30mm), and run the clipper through your whole head.
1b) If you want to style the top part of your hair (e.g., for parting, or styling), leave that part longer (or don't cut that at all) for later.
2) Pick a shorter one (e.g., 25cm) and run it along the areas close to the ear, and the back of your neck.
3) Repeat step 2, with even shorter length if desired. The key is to "blend" different lengths of hairs.
4) To add "layers" to your hair (to make your hair more "choppy" and to fine tune it to frame your face), use a pair of scissors, and cut "into" your hair like this.

Of course, if you need a really clean cut, or fancy style, you'll have to go to a hair salon.

Using BitSet or BitVector to store a set of integers

When programming with Java, we typically use HashSet to store a set of Integer, if we want to keep track of which Integer has been used and which hasn't. When we need to store millions of Integers, HashSet can be too slow, and it may take up too much memory.

Java's BitSet and BitVector from COLT are great alternative; they are MUCH faster and consumes way less memory. For example, to store 20 million integers, both BitSet and BitVector only use about 2.5MB and take only, respectively, 0.67s and 0.27s (which means BitVector is the fastest) to set and get all of the values once. using HashSet, it takes ~962MB, and 6.3s. This is more than 380 times of memory saving, and ~20 times speed up.

One "disadvantage" for BitVector is that it's size can't dynamically scale, but you can always fake it by manually growing or shrinking the set via setSize(...)

I can't figure out why Java's HashSet implementation uses so much RAM; I tried to reduce the size to 2 million, and that uses 100MB, approximately 1/10 of the size when having 20 millions. So this means indeed the Java implementation is memory-hungry. I'm sure there's some way to cut down the memory consumption, but I haven't spent the time to investigate.

Speed Comparison of: (1) Java's built-in HashMap, (2) Trove's TIntIntHashMap, and (3) Colt's OpenIntIntHashMap

Finally get to do a comparison among several popular Java HashMap implementations:

• Java's built-in HashMap
• Trove's implementation for primitive types
• Colt's implementation for primitive types

Not too surprisingly, Java's built-in implementation is the slowest, and consumes the largest amount of memory among the three, since it stores objects instead of primitive types (e.g., instead of storing the number 3 as an int, it stores it as an Integer). What WAS surprisingly, however, was that Colt's implementation was SO MUCH faster than Java's implementation. For example, for a int->int hashmap, running 10,000,000 puts() with Colt took less than 1/3 of the time. Running 10,000,000 gets() took less than 1/6, and 10,000,000 containsKey() less than 1/5! This is some substantial timing saving. This time-saving advantage is still there, even when we use a int->float[] hashmap. Memory-wise, the primitive implementations from Trove and Colt don't save us that much. For int->int hashmap, both Trove and Colt save us about 30% of memory, but for int->float[] (each float[] value has length 5) save us less than 10%. Nevertheless, there's still some saving. Since I need to deal with large data sets, where it's common to use hashmaps with millions of entries, I'll use the Colt implementation from now on, for both it's impressive time saving and (slighly) lower memory consumption. Here's the code I used for evaluating int->float[] hashmap. To get the timing and memory consumption for one implementation, comment out the other two that you DON'T want. You will also need to pass in these arguments to the VM, so enough heap memory will be alocated "-Xms1200M -Xmx1200M".

import java.util.HashMap;

import cern.colt.map.OpenIntIntHashMap;
import cern.colt.map.OpenIntObjectHashMap;
import gnu.trove.TIntIntHashMap;
import gnu.trove.TIntObjectHashMap;


public class Compare {
 
 public static void main(String args[]){
  
  System.out.println("1st line: time used(s)\n2nd line: heap memory used so far(MB)");
  
  int n = 10000000;
  
  long startTime = System.nanoTime(); 
  long startHeapSize = Runtime.getRuntime().freeMemory();

  
  // BEGIN: benchmark for Java's built-in hashmap
  System.out.println("\n===== Java's built-in HashMap =====");
  HashMap jIntIntMap = new HashMap();

  System.out.println("\n-- " + n + " puts(key, value) --");
  startTime = System.nanoTime(); 
  for (int i = 0; i < n; i++) { jIntIntMap.put(i,new float[]{0f,1f,2f,3f,4f}); }
  System.out.println( (System.nanoTime() - startTime) / 1000000000.0 );
  System.out.println( (startHeapSize - Runtime.getRuntime().freeMemory()) /1048576.0  );  
  
  System.out.println("\n-- " + n + " gets(key) --");
  startTime = System.nanoTime(); 
  for (int i = 0; i < n; i++) { jIntIntMap.get(i); }
  System.out.println( (System.nanoTime() - startTime) / 1000000000.0 );
  System.out.println( (startHeapSize - Runtime.getRuntime().freeMemory()) /1048576.0  );  

  System.out.println("\n-- " + n + " containsKey(key) --");
  startTime = System.nanoTime(); 
  for (int i = 0; i < n; i++) { jIntIntMap.containsKey(i); }
  System.out.println( (System.nanoTime() - startTime) / 1000000000.0 );
  System.out.println( (startHeapSize - Runtime.getRuntime().freeMemory()) /1048576.0  );  
  // END  
  
  
  // BEGIN: benchmark for Trove's TIntIntHashMap
  System.out.println("\n===== Trove's TIntIntHashMap =====");
  TIntObjectHashMap tIntIntMap = new TIntObjectHashMap();

  System.out.println("\n-- " + n + " puts(key, value) --");
  startTime = System.nanoTime(); 
  for (int i = 0; i < n; i++) { tIntIntMap.put(i,new float[]{0f,1f,2f,3f,4f}); }
  System.out.println( (System.nanoTime() - startTime) / 1000000000.0 );
  System.out.println( (startHeapSize - Runtime.getRuntime().freeMemory()) /1048576.0  );  
  
  System.out.println("\n-- " + n + " gets(key) --");
  startTime = System.nanoTime(); 
  for (int i = 0; i < n; i++) { tIntIntMap.get(i); }
  System.out.println( (System.nanoTime() - startTime) / 1000000000.0 );
  System.out.println( (startHeapSize - Runtime.getRuntime().freeMemory()) /1048576.0  );  

  System.out.println("\n-- " + n + " containsKey(key) --");
  startTime = System.nanoTime(); 
  for (int i = 0; i < n; i++) { tIntIntMap.containsKey(i); }
  System.out.println( (System.nanoTime() - startTime) / 1000000000.0 );
  System.out.println( (startHeapSize - Runtime.getRuntime().freeMemory()) /1048576.0  );  
  // END     
  
  // BEGIN: benchmark for Colt's OpenIntIntHashMap
  System.out.println("\n===== Colt's OpenIntIntHashMap =====");
  OpenIntObjectHashMap cIntIntMap = new OpenIntObjectHashMap();

  System.out.println("\n-- " + n + " puts(key, value) --");
  startTime = System.nanoTime(); 
  for (int i = 0; i < n; i++) { cIntIntMap.put(i,new float[]{0f,1f,2f,3f,4f}); }
  System.out.println( (System.nanoTime() - startTime) / 1000000000.0 );
  System.out.println( (startHeapSize - Runtime.getRuntime().freeMemory()) /1048576.0  );  
  
  System.out.println("\n-- " + n + " gets(key) --");
  startTime = System.nanoTime(); 
  for (int i = 0; i < n; i++) { cIntIntMap.get(i); }
  System.out.println( (System.nanoTime() - startTime) / 1000000000.0 );
  System.out.println( (startHeapSize - Runtime.getRuntime().freeMemory()) /1048576.0  );  

  System.out.println("\n-- " + n + " containsKey(key) --");
  startTime = System.nanoTime(); 
  for (int i = 0; i < n; i++) { cIntIntMap.containsKey(i); }
  System.out.println( (System.nanoTime() - startTime) / 1000000000.0 );
  System.out.println( (startHeapSize - Runtime.getRuntime().freeMemory()) /1048576.0  );  
  // END    
  
  
 }

}

Embedding web browser in Java application

I found this NativeSwing library an excellent library for embedding a web browser in a Java application, seemingly easier to use and much better documented that JDIC.

WPF: getting a data-bound data template and the items within it

I'm so happy to find out about Rich Strahl's post on how to get the items within a data template that is data-bound to, say, a list box. It turns out the only way to get an item inside a data template is to programmatically go into the UI hierarchy of the template... not elegant at all!

The best tutorial about data binding in WPF

It turns out that Microsoft's MSDN has the best tutorial around about how to do data binding in WPF. Everything is explained clearly; the diagrams, tables, and examples are just perfect. Double thumbs up. Waaaay better than many other online data binding tutorials.

The correct way to repaint a form or control in c#

There are two ways to repaint a form and its contents:

• You can use one of the overloads of the Invalidate method with the Update method.

• You can call the Refresh method, which forces the control to redraw itself and all its children. This is equivalent to setting the Invalidate method to true and using it with Update.

The Invalidate method governs what gets painted or repainted. The Update method governs when the painting or repainting occurs. If you use the Invalidate and Update methods together rather than calling Refresh, what gets repainted depends on which overload of Invalidate you use. The Update method just forces the control to be painted immediately, but the Invalidate method governs what gets painted when you call the Update method.

(Above info quoted from MSDN http://msdn2.microsoft.com/en-us/library/system.windows.forms.control.update.aspx)