Documentum Search – Lucene versus FAST

Interesting blog about indexing and searching with Lucene versus FAST.

http://blog.tsgrp.com/2010/03/17/documentum-search-–-lucene-versus-fast/

http://www.lucidimagination.com/blog/2010/03/19/actual-mileage-may-vary/

AspectJ In Action 2nd Edition

Hi All,

For those of you who are interested in learning or developing AOP with AspectJ or Spring AOP then I would definitely recommend reading AspectJ in Action 2nd Edition. I was asked to be a technical reviewer for the book and I have to say it is an extremely valuable reference for learning AOP. It has clear and excellent examples of enterprise Aspects that you could probably use for your projects straightway.

Most people think of tracing and profiling when they think of AOP however this book provides more usecases for AOP than the traditional areas of cross cutting concerns.

Good reading!

Hi

It's been a while since I posted anything, sorry about that. There are tons of things going on at the moment, with work and family. My wife had another baby girl so my hands are full with the baby and the toddler who wants to 60 million of things at the same time!

So I thought I'd post something today and tell folks what I've been up to. Well apart from slaving away at work I have been looking at designing and building a complete search solution. A complete search solution? What does that mean? The complete search solution is basically one search manager that allows you to d0 free text search across both documents and domain objects. The client isn't exposed to the details of the search and as a result the complexities of the domain/documents are hidden.

The technology stack I am using to do is as follows:

Java 6
Lucene 2.4
Hibernate Search
Hibernate Annotations
Spring 2.5
Spring MVC (I've had enough of Struts 2, the support from Struts mailing sucks!)

It's been fun working on this in my spare time. Last christmas I was sitting on the sofa (kids and wife asleep) thinking about search (as you do...) and thought it would be really cool to have one search api that would delegate the work of searching documents and domain objects. The manager would return summary objects which meant other search components could be added to the search manager.

The design looks like this:















The Search Manager has the following as the API:

SearchResponse search (SearchRequest searchRequest).

A simple method with a simple parameter object (SearchRequest). Now folks I want to emphasis that the SearchRequest is not a command object it is a parameter object which encapsulate properties that will be used to do the search. It's amazing how many silly command objects I am seeing at work and other places now a days. It's the return of the form beans (aaahh!!!).

Anyway...so where am I with the project. Well I have got full document indexing and search working using native Lucene. I have also completed domain search component using Hibernate Search. The final glue code the search manager is also completed. The search manager uses a SearchCollector class which collects all search components (classes that implement the Search interface). I want to use OSGi to dynamically load/unload search components at runtime. I have downloaded Spring DM server.

The search manager takes the collection of searchers and creates a Executor service and submits callable jobs to execute the search1es. The final results are collected and added to the SearchResponse which is returned to the client.

Some resources that I have used are:

Lucene in Action
Hibernate Search In Action
Spring Recipes
Java Persistence with Hibernate
AspectJ in Action

On a side note I just want to mention that the Lucene community have been absolutely great at helping me with loads of stuff (search code validation, highlighting summary, etc). If you are thinking of looking at Lucene then remember the mailing list is your friend.

I am going to modify my tests and project structure and then I will upload the project to Google. Andy, I know you want me to put the code as soon as I can but I would like to put the project on Google as a self contained project with no external dependencies. So people can check it out and run the tests easily. It won't be long.

I'd be happy to provide more information about this if people are interested. Unfortunately I've got to sign off. So as per usual please free to leave any comments. I know this post seems rushed but I'll had some code samples from my project soon.

Cheers
Amin

Hibernate Search and GigaSpaces XAP

For those who are interested I shall be posting the process of integrating Hibernate Search and GigaSpaces on my blog soon. I am on the final stages of completing the project which hopefully I shall make available later this week or eary next week (depending if my wife doesn't go into labour!)

Sorry for not putting anything more or interesting today!

Hibernate Search and Spring 2.5

Hi there,

After doing my first blog I have been excited to post something new.  So this blog entry is about Hibernate Search and integrating with Spring.  Hibernate Search (www.search.hibernate.org)is a cool technology that uses Apache Lucene under the hood to allow free text search across your domain model.  

I have been playing with Hibernate Search for some months now and I thought I'd share my code example with you.  By the way I am using Hibernate annotations, I haven't looked at doing it via XML.

Book.java

@Indexed

@Table(name="BOOK")

@Analyzer(impl=com.amin.spring.app.util.EnglishAnaylzer.class)

public class Book implements Serializable {

@Id @GeneratedValue(strategy=GenerationType.AUTO)

@Column(name="BOOK_ID")

@Type(type="java.lang.Long")

@DocumentId

private Long id;

@Column(name="BOOK_TITLE")

@Field(index=Index.TOKENIZED, store=Store.YES)

private String title;

@Column(name="BOOK_DESCRPTION")

@Field(index=Index.TOKENIZED, store=Store.YES)

private String description;

@Column(name="BOOK_CATEGORY")

@Field(index=Index.TOKENIZED, store=Store.YES)

@Enumerated(EnumType.STRING)

private CategoryEnum category;

//getters and setters

}

BookRepositoryImpl.java

@Repository

public class BookRepositoryImpl implements BookRepository {

   private HibernateTemplate hibernateTemplate;

   

   private static final String[] FIELD_NAMES = new String[]{"title", "description", "category"};

   @Autowired

   public void setHibernateTemplate(HibernateTemplate hibernateTemplate) {

     this.hibernateTemplate = hibernateTemplate;

   }

   

   public List search ( final String searchTerm) {

    List searchResult = (List) hibernateTemplate.execute( 

new HibernateCallback() {

public Object doInHibernate(org.hibernate.Session session) {

FullTextSession fullTextSession = Search.createFullTextSession(session);

QueryParser parser = new MultiFieldQueryParser( FIELD_NAMES, new StandardAnalyzer());

org.apache.lucene.search.Query query = parser.parse(searchString);

org.hibernate.Query hibernateQuery = fullTextSession.createFullTextQuery(query, Book.class);

List results = hibernateQuery.list();

return results;

}

}, true);

return searchResults

}

   }

Now I need to configure the hibernate session factory in Spring to enable Hibernate Search. The following provides a snippet what you will need to add to the session factory bean definition in the spring context file. 

spring-context.xml

<property name="eventListeners">

<map>

<entry>

<key>

<value>post-update</value>

<bean class="org.hibernate.search.event.FullTextIndexEventListener" />

</property>

Add the same FullTextIndexEventListener for post-insert and post-delete

This will enable automatic indexing of your domain object upon insert, update or delete.  You will also need to specify the location of where  the indexes should be created.  This can be added to the hibernateProperties section of the sessionFactory setup.  There are additional properties that you can add, for example you can enable asynchronous indexing. Please refer to the Hibernate Search documentation on the full properties that can be used.

Once you have got this example working you can use Luke (an Swing application located at www.getopt.org/like/) that allows you to view your indexes and also perform searches.  I found that using Luke was very handy when needing to view my indexes and build Lucene queries.

Now finally a testcase.  The test case is very similar to the one I posted last time but I've added a transactional context. 

BookRepositoryTest.java

Set up a testcase that saves a book to the database.  [One thing to note that the event listeners will only execute after the completion of a Hibernate transaction so Spring's transactional tests won't work unless the object is saved to the database.  You could create an object commit to the database, perform a search, do some assertions and then delete the object from the database.]

@Test

@Rollback(false)

public void testCanSearchWithResult() {

List books = bookRepository.search("xyz");

assertNotNull(books);

}

In the above example the search is performed on all fields (so the query looks something like 'title:xyz description:xyz' and so on), if however you wanted to search on a particular field (say 'title') then you could pass the string "title:XYZ" and search would only be done on only the title field.  There are various ways to construct a search query in Lucene and I recommend Lucene In Action as a good resource for getting to know more about Lucene.

Hibernate Search documentations provides good information about embedded objects and how to set up the indexing processes on these objects.  I also found the Hibernate Search forum very helpful when trying to find solutions to problems.

At the end of the day you can still get to the Lucene api from Hibernate Search.  With Hibernate Search you get the added advantage that when you perform a query you get domain objects back without having to do any conversions from Lucene documents.

That's it.  Hope this provides enough information to get you started.   As I mentioned last time your comments are welcomed.

@Configurable example with Spring 2.5

Hey,
My first blog! I feel somewhat late on the blogging scene...but better late than never!

I recently did a simple (maybe too simple..i hope to add more features) example of using Spring 2.5 @Configurable. The example shows how to configure your domain objects (normally outside the control of the container,for example objects returned by Hibernate) so that dependencies can be injected by Spring.

Now I know there are a lot of opinions on why you would do this (Personally I like the idea of having DomainServices which aren't related to application services or infrastructure services). We can have this debate later but for those of you who are interested in doing this here are the following code samples.


DomainObject.java [package com.amin.spring.app.domain]

@Configurable(dependencyCheck=true)

public class DomainObject {

private String name;

private DomainService domainService;

@Autowired

public void setDomainService(DomainService domainService) {

this.domainService = domainService;

}

public DomainService getDomainService() {

return domainService;

}

public void setName(String name) {

boolean isValidName = domainService.isValidUserName(name);

if (!isValidName) {

throw new UsernameValidationException("username already used. please try alternative");

}

this.name = name;

}

public String getName() {

return this.name;

}

}

DomainService.java [package com.amin.spring.app.service]

public interface DomainService {

public boolean isValidUserName(String name);

}

DomainServiceImpl.java[package com.amin.spring.app.service] [the implementation class is simple! But it's to give you an idea]

@Service

public class DomainServiceImpl implements DomainService {

public boolean isValidUserName(String name) {

boolean isValidUserName = StringUtils.hasText(name); return isValidUserName;

}

}

conf/spring/bean.xml [snippet excluding xml declaration..be sure to use 2.5]

<context:annotation-config />

<context:spring-configured />

<context:load-time-weaver />

<context:component-scan base-package="com.amin.spring.app" />

Amazing...that's it!

DomainObjectTest.java [src/test/java - com.amin.spring.app.domain]

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(locations={"/conf/spring/bean.xml"})
public class DomainObjectTest {

@Test
public void testServiceInjectionToDomainObject() throws Exception {
DomainObject domainObject = new DomainObject();
assertNotNull("Domain Service was not set", domainObject.getDomainService());
}

@Test
@ExpectedException(value=java.lang.RuntimeException.class)
public void testServiceInjectionToDomainObjectWithValidationException() throws Exception {
DomainObject domainObject = new DomainObject();
domainObject.setName(null);
assertNotNull("Validation Service was not set", domainObject.getDomainService());

}
}


Right...for this example I am using Load Time Weaving (you can do with build time weaving with Maven or Ant) which will enable AspectJ to weave classes that are marked with @Configurable, allowing Spring to inject any dependencies into domain object before it is returned to the calling code. In order to do this you will need to configure the run configuration of the testcase (i do this via eclipse). This can be done by adding the following to the JVM argument:

-javaagent:/spring-agent.jar

And then you're ready to go....

Feel free to comment on the blog and/or on the example code.