Indexes

Indexing is a strategy for expediting retrievals from a database by storing values in a separate table. This section describes how Dari implements indexing.

Dari implements indexing by requiring searches to be performed on indexed fields; searches are not allowed on non-indexed fields. You declare an indexed field by applying the @Recordable.Indexed annotation.

import com.psddev.dari.db.Record;

public class User extends Record {

    @Indexed 
    private String userName;

    private String screenName; 

}

Given the field definitions in the previous snippet, you can search for specific instances of User using the Query#where method on the indexed field.

List<User> = Query.from(User.class).where("userName = 'zombie'").selectAll();

The previous snippet returns all User records with userName zombie.

In contrast, you cannot search for specific instances of User records using the Query#where method on a non-indexed field.

List<User> = Query.from(User.class).where("screenName = 'vampire'").selectAll();

The previous snippet returns an error because the field screenName is not indexed.

Keep in mind the following when indexing fields:

• By default, fields are not indexed. You must explicitly apply the annotation @Indexed to a field for Dari to index it.
• You cannot apply @Indexed to fields with the modifier transient.

See also:

• Indexed methods
• Reindexing existing objects
• Strategies for effective indexing
• Visibility labels and selection fields
• Visibility labels
• Database index at Wikipedia

Dari can index the results of methods, so you can quickly retrieve all of the records that generate a particular result.

Indexing methods in self-contained objects

Some objects in Brightspot contain all of the properties that describe the entire object, and you can retrieve the entire object with a single database lookup. For example, if an article object has a headline and an embedded author, then when you retrieve the article you also retrieve the entire author object. (To index methods that use referred objects, see "Indexing methods having referred objects.")

In the following example, the getFullName method is indexed. When a User object is created, Dari automatically executes the method and stores the result in an index.

public class User extends Record {

    private String firstName;
    private String lastName;

    public String getFirstName() {
        return firstName;
    }

    public String getLastName() {
        return lastName;
    }

    /*
     * Use getFirstName and getLastName to return
     * a user's full name. The @Indexed annotation
     * indexes all results of this method.
     */
    @Indexed
    public String getFullName() {
        return (getFirstName() + " " + getLastName());
    }

}

You can use the method getFullName to search for instances of User having a specific full name.

List<User> = Query.from(User.class).where("getFullName = 'John Adams'").selectAll();

The previous snippet returns all User records whose full name is John Adams.

You can apply the @Indexed annotation to methods with the following characteristics:

• Identifier starts with a get, is, or has literal, such as getName, isActive, or hasEntry.
• Returns a value (does not return void).
• Has public access.

See also:

• Indexed fields
• Reindexing existing objects
• Strategies for effective indexing
• Embedded objects

Indexing methods having referred objects

Some objects in Brightspot contain properties that refer to other objects. For example, if an article object has a referred tag, then determining the tag’s name requires a secondary retrieval. You can index methods that resolve such references. (To index methods that use embedded objects, see "Indexing methods in self-contained objects.")

In the following example, the getTagName method is indexed. When an Article object is created, Dari looks up the associated Tag, retrieves the tag’s name, and adds that name to the index of all such tag names.

public class Article {

    private Tag tag;

    public Tag getTag() {
        return tag;
    }

    /*
     * Use resolveAndGet to a) retrieve an article's referenced tag object and
     * then b) return the tag's name.
     */
    @Indexed
    public String getTagName() {
        return Optional.ofNullable(StateUtils.resolveAndGet(this, Article::getTag))
                .map(Tag::getName)
                .orElse(null);
    }
}

You can use the method getTagName to search for instances of Article having a specific tag name.

List<Article> = Query.from(Article.class).where("getTagName = 'Morning Briefing'").selectAll();

The previous snippet returns all Article records having the tag Morning Briefing.

You can apply the @Indexed annotation to methods with the following characteristics:

• Identifier starts with a get, is, or has literal, such as getName, isActive, or hasEntry.
• Returns a value (does not return void).
• Has public access.

See also:

• Indexed fields
• Reindexing existing objects
• Strategies for effective indexing
• Object references

Indexing a database requires additional storage and processing time beyond that required to save a record. Every time you instantiate a new object with an indexed field, Dari stores the object itself and then adds a new index entry for each indexed field; similar operations occur when you modify an indexed field’s value, modify the value of a field used in an indexed method, and delete an object with indexed fields. For example, referring to the snippet "Class with indexed field," when you create a new instance of User—

• Dari stores the new instance in the database.
• Dari creates a new index entry for the field userName.

The extra processing required for indexing impacts response times as the size of table holding the affected class grows. As a result, a common best practice is to index only those fields and methods you expect will be used in where clauses.

A visibility label is a convenience property with the annotation @Indexed(visibility = true). This annotation excludes a record from being included in a query—even if the record satisfies the query’s predicate condition. If the annotated field’s value is non-null, Dari excludes the record from the results.

Visibility labels are useful when you normally do not want to retrieve all records matching search criteria. Examples include the following:

• Displaying all articles by an author except those that are in draft status.
• Displaying all comments associated with an article except those that were not approved.

You can exclude such records without a visibility label, but you need to remember to include an additional condition each time you execute the query. With a visibility label, the default is to exclude the matching records, and you override the exclusion as needed. For details, see Retrieving records with visibility labels.

import com.psddev.dari.db.Content;

public class Comment extends Content {

    @Indexed(visibility = true)
    private Boolean waitingApproval approved; 

}

Given the previous snippet, the following query excludes those comments for which waitingApproval is non-null.

List<Comment> = Query.from(Comment.class).selectAll();

The following table summarizes the behavior of visibility labels.

The section Excluding records from queries with visibility labels describes how to exclude records from a query that otherwise satisfy a predicate condition. The techniques described in this section describe how you can retrieve those excluded records.

Retrieving records by visibility label

You can retrieve records excluded by a visibility label by examining the value of the visibility label itself. Suppose you have a comment instantiated from the snippet "Class With @Indexed(visibility)," and you perform the following query:

List<Comment> article = Query.from(Comment.class)
    .where("waitingApproval = true")
    .selectAll();

The previous snippet retrieves all comments having a visibility label’s value of true. Because true is a non-null value, these comments are normally excluded from search results, but they are included in this form of a query.

Retrieving records with fromAll

The method fromAll returns all records of all types regardless of a visibility label’s effect. You can use this to isolate objects of a given type regardless of the value of the visibility label.

Retrieving a single record

You can retrieve a single record excluded by a visibility label using the method fromAll() followed by selecting on the _id field. Suppose you have a comment instantiated from the snippet "Class With @Indexed(visibility)," and you perform the following query:

Comment comment = Query.fromAll().where("_id = '123456789'").first();

The previous snippet retrieves a comment with ID 123456789 even if the visibility label approved is not null.

Retrieving multiple records

You can collect all objects of a particular content type into a list regardless of the value of each object’s visibility label. Suppose you have comments instantiated from the snippet "Class With @Indexed(visibility)." You can query for all objects and then isolate the comments—even if they are hidden by a visibility label.

List<Object> everything = Query.fromAll()
    .where("cms.content.updateDate > ?", sevenDaysAgo)
    .selectAll();

List<Comment> comments = everything
    .stream()        
    .filter(c -> c instanceof Comment)
    .map( c -> (Comment) c)
    .collect(Collectors.toList());

The previous snippet retrieves all objects in the Brightspot project within a given date range, and then iterates over them to collect the comments into a list. In this scenario the visibility label waitingApproval has no effect.

Applying the @Indexed annotation, building, and redeploying your project does not automatically index existing objects. If you apply an @Indexed annotation to fields and there are objects with those fields already in the database, you need to manually reindex the existing objects. For details, see Reindexing.