Blog moved to new site

Blog moved to new site: http://artemgolubev.com

Approaches to search ZIP codes within given radius

There are several approaches to get zip codes within given radius from given zip:

1. Round circle by square if zips given by points;


2. Exact 3D geometry calculate if zips given by points;


3. Exact GIS calculate if zips given by polygons.

Approximation by square

We approximate circle by square on surface, where:

1. latitude +- radius/EARTH_RADUIS


2. longitude +- radius/(EARTH_RADUIS*cos(latitude))

cos here is because longitudes distances differs depending on latitudes (say, in polar it is 0).

The databases for 1999 is accessible open source, but up to date database costs money. Free database:
http://svn.baconbear.com/rails_plugins/acts_as_locateable/trunk/data/zip_code_data.csv

Open source database is about 42 000 records. Up to date is about 72 000. See, for instance:

1. http://www.zip-codes.com/


2. http://www.zip-code-database.org/


3. http://www.google.com/search?q=zip+codes+database

Exact 3D calculations

3D coordinates:

1. x= EARTH_RADIUS*sin(latitude)*cos(longitude)


2. y= EARTH_RADIUS*sin(latitude)*sin(longitude)


3. z= EARTH_RADIUS*cos(latitude)

Search points must be within sphere:

x*x + y*y + z*z <= radius*radius

It is non-efficient table full-scan implementation.

See http://en.wikipedia.org/wiki/Spherical_coordinate_system

Exact zip polygon databases

Here you have to install open-source PostGIS (http://postgis.refractions.net/) system on your PostgreSQL and buy zip codes polygon database.
After installing PostGIS you can query zip codes within given radius the next way (http://postgis.refractions.net/pipermail/postgis-users/2007-February/014760.html):

SELECT z.zipcode FROM zipcode z, zipcode z2 WHERE z2.zipcode = '02109' AND expand(z2.the_geom, "whatever radius you want goes here but your spatial ref needs to be in that metric or you need to convert the radius") && z.the_geom AND distance(z.the_geom, z2.the_geom) <= "radius"


Zip codes polygon databases:

1. http://maps.huge.info/blog/2007/04/zip_code_polygon_databases.html


2. http://www.google.com/search?q=zip+codes+poligon+database

Creating Web 2.0 RIA applications with Flex and Rails

Flex now is much for web development: we have OOP there, fine web-services integration, code and layout separation and more.
I personally think that the best way of creating web application is designing (or modeling) how it’ll look like for end-user, what and what it going to afford. So, let us think we already have some Flex web-application and going to connect it to some web-server to make it do something meaningful.
We have at least 4 options here, we can use:

1. Custom XML-based HTTP service;


2. Custom non-XML HTTP service;


3. SOAP-based web-service;


4. AMF (Adobe proprietary) based service.

1. Custom XML-based HTTP service

Flex will send HTTP request and receive XML answer like following:

<?xml version="1.0" encoding="UTF-8" ?>
<list>
  <item product="Computer">
    <UID>12321</UID>
    <CPU>4Gh</CPU>
    <RAM>2Gb</RAM>
    <HDD>400Gb</HDD>
  </item>
</list>

Pro here is that you can do comparatively small XML here (smaller that SOAP one).
Contra is that you are responsible for constricting XML on the server side and working with it (say, using DOM) on the client side.
It is rather inconvenient to do this additional work dealing with some XML structures, but many systems working just the way I described here.

2. Custom non-XML HTTP service

Similar to previous approach, but here you deal with yourself designed metadata like:

list[
  Computer[
    UID=12321
    CPU=4Gh
    RAM=2Gb
    HDD=400Gb
  ]
]

It can be not so verbose as XML, but you now responsible for parsing your custom structures yourself. It has so many troubles that it even does not worth to be considered, unless you write system like AMF yourself.

3. SOAP-based web-service

Absolutely pain-free method, since now nearly all web-application frameworks has transparent built-in methods for SOAP based web-services. So does Flex. Troubles could begin here than you will experience high loads with crowds of users. Constructing and parsing big XMLs will kill your web-servers and channels. SOAP does best with heterogeneous systems integration like those used in b2b, but it is too heavy for ordinary web applications.

4. AMF-based service

Pros here comes form the nature of the protocol. It is fast binary over HTTP. Designed just for Rich Internet Applications. Why not to use proprietary protocol for communication with server if you already chosen proprietary Flex system to be used? The reasons could be: you don’t know how to integrate it in your server and you are afraid of possible troubles by dealing with proprietary technology. Here in this article we will consider both questions.
Let us be more exact. There are 2 versions of the protocol: AMF3 for Flex and AMF0 for Flash. So let us focus on AMF3 only (further in this article I’ll use AMF in meaning AMF3).

There are some frameworks for AMF: Adobe Flex Data Services (http://www.adobe.com/products/flex/dataservices/), AMFPHP (http://amfphp.org/), WebORB (http://www.themidnightcoders.com/weborb/). May be there are some others. Frameworks are available for .NET, Java, PHP and Ruby on Rails. But I’ve found the only one available for Rails. It is WebORB. So further in this article I’ll describe my experience about dealing with WebORB plugin for Rails.

Flex and Rails

Good news is that Flex applications works fine with Rails via WebORB (see http://www.themidnightcoders.com/weborb/rubyonrails/gettingstarted.htm how to use it). Bad one is that it conflicts with number of other plugins like act_as_attachment (we wrote workarounds by changing the WebORB plugin).

Sessions

Another inconveniency is that WebORB service classes which you write for your web services are plain classes, not inherited anything. So you don’t have those nice features like ones you have for your controllers or SOAP web-services. Therefore, your services are stateless by default, since they do not have access to the session object (HTTP session maintained via cookie). We wrote some utility classes and slightly changed WebORB to add session object. But you can choose to maintain session on the client side by SharedObject Flex class: http://livedocs.adobe.com/flex/2/langref/flash/net/SharedObject.html

File upload

There are no obvious ways to upload file via AMF. The only way to upload by the Flex we’ve found was using FileReferenсe + URLRequest objects. There is some obstacle here. The URLRequest initialize another HTTP Session than AMF one’s. Therefore we can't identify user who make URLRequest on the server side. The workaround for this problem is to send additional information with URLRequest to make possible to identify user. Another inconvenience is that URLRequest can't get more than http status code as result (no messages or values).

Search Engine Optimization and mobile access

Search Engine Optimization (SEO) stays for providing readable HTML data for search engines like google.com, yahoo.com, and others. Usually the main argument against using Flash/Flex is absence of SEO possibility. But there is workaround for this issue. The trick here is that you can create simple xHTML site with all SEO information and then user load such page JavaScript will load your Flex on corresponding page. The big advantage of this approach is that it automatically makes your site available for mobile devices (WAP 2.0) if you use xHTML Mobile Profile DTD.

Types mapping

By tests we've found that it is the next (Flex <-> Ruby):

1. int <-> Fixnum


2. Numeric <-> Float


3. Date <-> Time


4. String <-> String


5. Boolean <-> Boolean

Be aware, that if you choose ":decimal" type for database column it will not be mapped to convertable Ruby type and therefore not accessible in Flex.

There is interesting site concerning Flex and Ruby questions: http://flexonrails.net/. I could recommend you to look throw this article: http://www.flex888.com/2007/09/20/10-flex-and-ruby-on-rails-integration-examples.html
and this one http://webddj.sys-con.com/read/295396.htm

Good luck!

Rails does not like slaves

I played with Ruby on Rails today and found that it can't convert slave to it's plural form slaves.
It automatically changes letter v to f in slave word and f to v in slafe word.

That was funny!

Get rid of your nested if operators!

Every one of you know how it is disappointing that you can not use

switch

operators with strings, doubles, your custom objects, etc. But anyway sometimes it is possible to get rid of your nested ifs if you will use static maps. Let us consider an example: here we need to set JavaBean properties of certain types:

 protected static Object fillTestBean(Object bean) throws InvocationTargetException, IllegalArgumentException, IllegalAccessException {
  Method[] methods = bean.getClass().getMethods();
  
  for (Method method : methods) {
   if (RTTIUtils.isSetter(method)) {
    Class type = RTTIUtils.getSetterType(method);
    if (type == String.class)
     RTTIUtils.invokeSetter(bean, method, testStr);
    else if (type == boolean.class)
     RTTIUtils.invokeSetter(bean, method, testBoolean);
    else if (type == byte.class)
     RTTIUtils.invokeSetter(bean, method, testByte);
    else if (type == char.class)
     RTTIUtils.invokeSetter(bean, method, testChar);
    else if (type == short.class)
     RTTIUtils.invokeSetter(bean, method, testShort);
    else if (type == int.class)
     RTTIUtils.invokeSetter(bean, method, testInt);
    else if (type == long.class)
     RTTIUtils.invokeSetter(bean, method, testLong);
    else if (type == float.class)
     RTTIUtils.invokeSetter(bean, method, testFloat);
    else if (type == double.class)
     RTTIUtils.invokeSetter(bean, method, testDouble);
    else if (type == Boolean.class)
     RTTIUtils.invokeSetter(bean, method, testBooleanObj);
    else if (type == Byte.class)
     RTTIUtils.invokeSetter(bean, method, testByteObj);
    else if (type == Character.class)
     RTTIUtils.invokeSetter(bean, method, testCharacterObj);
    else if (type == Short.class)
     RTTIUtils.invokeSetter(bean, method, testShortObj);
    else if (type == Integer.class)
     RTTIUtils.invokeSetter(bean, method, testIntegerObj);
    else if (type == Long.class)
     RTTIUtils.invokeSetter(bean, method, testLongObj);
    else if (type == Float.class)
     RTTIUtils.invokeSetter(bean, method, testFloatObj);
    else if (type == Double.class)
     RTTIUtils.invokeSetter(bean, method, testDoubleObj);
    else if (type == Date.class)
     RTTIUtils.invokeSetter(bean, method, testDate);
   }
  }

  return bean;
 }

(here all variables starting with test are constants)
Seems really annoying to write all this ifs and, also what if we will need to check values? Another function will also need this nested ifs!
In this case I usually do the following:

 private static final Map initObjects = new HashMap();
 private static boolean isCosideredType(Class type) {
  return initObjects.keySet().contains(type);
 }
 static {
  initObjects.put(String.class, testStr);
  initObjects.put(boolean.class, testBoolean);
  initObjects.put(byte.class, testByte);
  initObjects.put(char.class, testChar);
  initObjects.put(short.class, testShort);
  initObjects.put(int.class, testInt);
  initObjects.put(long.class, testLong);
  initObjects.put(float.class, testFloat);
  initObjects.put(double.class, testDouble);
  initObjects.put(Boolean.class, testBooleanObj);
  initObjects.put(Byte.class, testByteObj);
  initObjects.put(Character.class, testCharacterObj);
  initObjects.put(Short.class, testShortObj);
  initObjects.put(Integer.class, testIntegerObj);
  initObjects.put(Long.class, testLongObj);
  initObjects.put(Float.class, testFloatObj);
  initObjects.put(Double.class, testDoubleObj);
  initObjects.put(Date.class, testDate);
 }
 
 protected static Object fillTestBean(Object bean) throws InvocationTargetException, IllegalArgumentException, IllegalAccessException {
  Method[] methods = bean.getClass().getMethods();
  
  for (Method method : methods) {
   if (RTTIUtils.isSetter(method)) {
    Class type = RTTIUtils.getSetterType(method);
    if (isCosideredType(type)) {
     RTTIUtils.invokeSetter(bean, method, initObjects.get(type));
    }
   }
  }

  return bean;
 }

Algorithm for generating serialVersionUID for java entities

I could suggest the following way for generating serialVersionUID for java entities.
It is staible in sense that it depends only on your schema, not JVM/time or date of creation.
On other hand it depends on all fields in entity.

If we have next n fields:
typeName_1 fieldName_1;
typeName_2 fieldName_2;
...
typeName_n fieldName_n;

we could calculate
serialVersionUID =
(fieldName_1.hashCode() XOR typeName_1.hashCode()) * 31^0+
(fieldName_2.hashCode() XOR typeName_2.hashCode()) * 31^1 + ... +
(fieldName_n.hashCode() XOR typeName_n.hashCode()) * 31^n

where fieldName_i.hashCode() is hashCode() of fieldName_i string
(for int id; it will be "id".hashCode())
and typeName_i.hashCode() is hashCode() of typeName_i string
(for int id; it will be "int".hashCode())

so, we will be independent from
1) JVM
2) time/date of creation
and depend on
1) fields order
2) field types
3) field names

How to override equals() and hashCode() of persistent objects using ids

The problem described in details in http://www.hibernate.org/109.html
The problem is that if we implement equals()/hashCode() as comparison and hashCode() of entity's ids (see http://djeang.blogspot.com/2005/08/override-equals-and-hashcode-methods.html) then the hashCode() (and, possibly, equals()) will change after saving entities if ids are generated by DB.
To solve this issue I can suggest to use inner classes with equals()/hashCode(). It will not affect Hibernate's internal collections taskflow, but will allow you to use ids in equals()/hashCode() after entities are persisted. On other hand hashCode() and equals() will not change in your persistent objects after your saved it.

public class Test3 {
   static class SomeEntity {
       private Integer id;

       private String data;

       public SomeEntity() {
           super();
       }

       public SomeEntity(Integer id, String data) {
           super();
           this.id = id;
           this.data = data;
       }

       public Integer getId() {
           return id;
       }

       public void setId(Integer id) {
           this.id = id;
       }

       public String getData() {
           return data;
       }

       public void setData(String data) {
           this.data = data;
       }

       public class SomeEntityId {
           private SomeEntity link = null;

           public SomeEntityId(SomeEntity link) {
               super();
               this.link = link;
           }

           @Override
           public boolean equals(Object obj) {
  if (this == obj)
   return true;
  if ((obj == null) || (obj.getClass() != this.getClass()))
   return false;
  if (this.link.getClass() != ((SomeEntityId) obj).getLink().getClass())
   return false;

               Integer thisId = link.getId();
               Integer otherId = ((SomeEntityId) obj).getLink().getId();
               if ((thisId == null) || (otherId == null))
                   throw new RuntimeException("Id not defined!");

               return thisId.equals(otherId);
           }

           @Override
           public int hashCode() {
               Integer thisId = link.getId();
               if (thisId == null)
                   throw new RuntimeException("Id not defined!");

               return thisId.hashCode();
           }

           public SomeEntity getLink() {
               return link;
           }
       };

       public SomeEntityId getDTO() {
           return new SomeEntityId(this);
       }
   }

   public static void main(String[] args) {
       SomeEntity e0 = new SomeEntity(null, "no id");
       SomeEntity e1 = new SomeEntity(1, "e1");
       SomeEntity e2 = new SomeEntity(2, "e2");
       SomeEntity e3 = new SomeEntity(3, "e3");
       SomeEntity e33 = new SomeEntity(3, "e33");

       Set aSet = new HashSet();
       // aSet.add(e0.getDTO());
       aSet.add(e1.getDTO());
       aSet.add(e2.getDTO());
       aSet.add(e3.getDTO());
       aSet.add(e33.getDTO());

       System.out.println("set size: " + aSet.size());
       System.out.println("set:");
       for (SomeEntity.SomeEntityId seId : aSet) {
           SomeEntity se = seId.getLink();
           System.out.println("entity: " + se.getData());
       }
   }
}

I even could suggest yo add it to entities code generation in such tools as Hibernate Tools.
I would like to mention also that SomeEntityId object could be stored in transient field of an entity.

Comments are wellcome.