Lift

Mapper is one of two persistence layers included with Lift (the other being Record). Mapper is an ORM system for relational databases that lets query your database and represents your data in Scala objects.

This is a very long page (perhaps soon to be many pages), so the following outline may prove useful.

• A Mapper Example
• Setting Up the Environment
  • Creating a Database Connection
  • Schemifier
• Creating a Record
• Transactions
• Querying the Database
• Fields
  • Overriding Field Settings
  • Predefined Types
  • Making Your Own Field Types
  • Validations
  • Filtering
• Relationships
  • OneToMany
    • OneToOne
  • ManyToMany
• Various Helper Traits
  • IdPK
  • MegaProtoUser
    • The Logged-In User
    • OpenID
  • CreatedUpdated
  • Others
• Pagination
• Tips
  • Controlling scale of MappedDecimal

A Mapper Example

A simple mapper model might be saved to src/main/scala/net/liftweb/modelexample/model/Post.scala and look like so:

package net.liftweb.modelexample {
package model {

import net.liftweb.mapper._

class Post extends LongKeyedMapper[Post] {
  def getSingleton = Post
  
  def primaryKeyField = id
  object id extends MappedLongIndex(this)
  object title extends MappedString(this, 140)
  object contents extends MappedText(this)
  object published extends MappedBoolean(this)
}

object Post extends Post with LongKeyedMetaMapper[Post]

}
}

As you can see, we’ve defined a model for a simple blog post, which we’ll extend it as we go through this article. For now you should notice that we have both a class and a companion object, to which we have mixed similarly named Mapper traits. Notice that the companion object’s trait has ‘Meta’ in its name. This is a convention that is followed throughout Mapper.

We put all the fields in the class, not in the companion object. This makes sense, as each blog post should have its own unique title and contents! They are objects, not vals or vars for reasons related to Scala’s internals. Make sure to always declare fields as objects within the model class and you’ll be fine.

MappedString, MappedText, and MappedBoolean all extend MappedField. We will go into other subclasses of MappedField and creating your own subclasses later in this article. As always, feel free to refer to the Scaladocs for more information and other subclasses.

Setting Up the Environment

Creating a Database Connection

You may have multiple database connection types, but normally you’ll just want to be connected to one database. So, you will need to define a database ConnectionManager and mark it as the default one. In the boot method of Boot.scala you might have the following lines:

import net.liftweb.mapper.{DB, DefaultConnectionIdentifier} 
DB.defineConnectionManager(DefaultConnectionIdentifier, myDBVendor)

But what is your actual database vendor? It’s a singleton object that extends ConnectionManager that you might as well also define in Boot.scala. Here is an example taken from one of the Lift app archetypes:

import net.liftweb.mapper.{ConnectionIdentifier, ConnectionManager, Schemifier}
import java.sql.{Connection, DriverManager}
import net.liftweb.common.{Box, Empty, Full}
import net.liftweb.util.Props

object myDBVendor extends ConnectionManager {
  private var pool: List[Connection] = Nil
  private var poolSize = 0
  private val maxPoolSize = 4
 
  private lazy val chooseDriver = Props.mode match {
    case Props.RunModes.Production => "org.apache.derby.jdbc.EmbeddedDriver"
    case _ => "org.h2.Driver"
  }
  
  private lazy val chooseURL = Props.mode match {
    case Props.RunModes.Production => "jdbc:derby:lift_mapperexample;create=true"
    case _ => "jdbc:h2:mem:lift_mapperexample;DB_CLOSE_DELAY=-1"
  }

  private def createOne: Box[Connection] = try {
    val driverName: String = Props.get("db.driver") openOr chooseDriver
    val dbUrl: String = Props.get("db.url") openOr chooseURL
 
    Class.forName(driverName)
 
    val dm = (Props.get("db.user"), Props.get("db.password")) match {
      case (Full(user), Full(pwd)) =>
        DriverManager.getConnection(dbUrl, user, pwd)
 
      case _ => DriverManager.getConnection(dbUrl)
    }
 
    Full(dm)
  } catch {
    case e: Exception => e.printStackTrace; Empty
  }
 
  def newConnection(name: ConnectionIdentifier): Box[Connection] =
    synchronized {
      pool match {
        case Nil if poolSize < maxPoolSize =>
          val ret = createOne
          poolSize = poolSize + 1
          ret.foreach(c => pool = c :: pool)
          ret
 
        case Nil => wait(1000L); newConnection(name)
        case x :: xs => try {
          x.setAutoCommit(false)
          Full(x)
        } catch {
          case e => try {
            pool = xs
            poolSize = poolSize - 1
            x.close
            newConnection(name)
          } catch {
            case e => newConnection(name)
          }
        }
      }
    }
 
  def releaseConnection(conn: Connection): Unit = synchronized {
    pool = conn :: pool
    notify
  }
  
}

As you can see, you have complete control over which database is used, how connections are handled (included pooled), and so on. You’ll notice that we retrieve various property values to set connection properties, meaning that it’s very easy to have different settings for different machines or circustances. For instance, the sample above used the Derby database when in the Production run mode but the in-memory H2 database in all other circumstances. See the Properties page for more information.

Schemifier

Schemifier can be used to create the database tables needed to store your model records. It will only add missing tables or columns so it is generally safe to use anytime, but care should still be taken with production data – it does not do database migrations.

For instance, we could call it in the boot method of Boot.scala like so:

Schemifier.schemify(true, Schemifier.infoF _, Post)

The first argument is documented as really write the schema update? , the second argument is a log function to be used by Schemifier . The final argument of schemify can be a variable number of Mapper models, so you can simply keep adding additional Mapper models to the end of the method call.

Creating a Record

If the model class instances are where the actual record data resides, what use is the companion object? Why, for actually creating, retrieving, and deleting these instances.

Let’s create a post and save it to the database:

import  net.liftweb.modelexample.model.Post
val myPost: Post = Post.create
myPost.title("My First Blog Post")
// MappedFields return the model instance, so we can chain assignments
myPost.contents("This is a very short blog post but it will have to do for now.").published(true)
val saved: Boolean = myPost.save

As you can see, new Post model instances are created by calling the create method on the Post companion object. Do not create a new instance using the new keyword (e.g. new Post). This is because Mapper needs to do a bunch of things behind the scenes when creating a new model instance.
The Boolean return value of save has historical reasons. It can be safely ignored according to David Pollak in the discussion “Meaning of the Return Value of Mapper.save?”.

Transactions

import net.liftweb.mapper.{By, DB}
import net.liftweb.db.DefaultConnectionIdentifier

DB.use(DefaultConnectionIdentifier) {connection => {
  User.findAll( By(...) )
  User.my_value(new_val).save()
}}

DB.use(DefaultConnectionIdentifier) {
   conn => DB.prepareStatement("INSERT INTO users (name) VALUES ('Bob');", conn) {
              st => st.executeUpdate()
   }
}

Can be nested. See Mailing list: Transactions with Mapper.

Querying the Database

At its most basic, we can retrieve all the posts:

val posts: List[Post] = Post.findAll

The find method will return the first post it can find, if there is one:

val post: Box[Post] = Post.find

Select just one field

Sometimes you want to only retrieve some fields of the posts, such as only the title:

import net.liftweb.mapper._
val publishedPosts: List[Post] = Post.findAllFields(Seq[SelectableField](Post.title))

findAllFields is equivalent to listing the column names after a SELECT SQL statement. In fact, the previous command uses the following SQL:

SELECT title FROM post;

Query Parameters

However, sometimes you want to only retrieve some of the posts, such as only the published ones:

import net.liftweb.mapper.By
val postsTitles: List[Post] = Post.findAll(By(Post.published, true))

By is equivalent to an equality test in the WHERE clause of an SQL statement. In fact, the previous command uses the following SQL, assuming the database supports boolean types:

SELECT * FROM post WHERE post.published = true;

There are plenty of QueryParams that you can use to construct your query and organize its results (though By is technically not a QueryParam). For instance, we can get all published blog posts and sort the results by their titles in ascending order:

import net.liftweb.mapper.{OrderBy, Ascending}
val publishedPosts: List[Post] = Post.findAll(By(Post.published, true), OrderBy(Post.title, Ascending))

There are times you may want to use wildcards in your queries, for example, if you are trying to run a query like:

SELECT * FROM post WHERE post.title LIKE "This is awe%"

you can do something like:

import net.liftweb.mapper.Like
val filteredPosts: List[Post] = Post.findAll(Like(Post.title, "This is awe%"))

Here’s a list of some useful QueryParams:

• ByList
• BySql
• Distinct
• MaxRows
• OrderBy
• PreCache

Check the Scaladocs for more.

Converting List[Post] to List[String]

On all the examples on this wiki you end up with a List of objects, but there are times that what you need is a List of strings. There is an easy way to convert them:

import net.liftweb.mapper.Like
val titlePost: List[String] = Post.findAllFields(Seq[SelectableField] (Post.title), 
          Like(Post.title, "This is%")
      ).map(_.title.is)

Fields

The Mapper fields are how complex Scala types are translated into (often generic) types that your underlying database can understand. Accordingly, they do a lot and proper use of them can greatly reduce complexity and lines of code in your Lift app.

Overriding Field Settings

You can override many of a field’s settings. For instance, let’s index the title of the Post and give it a default value of “New Blog Post”:

class Post extends LongKeyedMapper[Post] {
  def getSingleton = Post
  
  def primaryKeyField = id
  object id extends MappedLongIndex(this)
  object title extends MappedString(this, 140) {
    override def dbIndexed_? = true
    override def defaultValue = "New Blog Post"
  }
  object contents extends MappedText(this)
  object published extends MappedBoolean(this)
}

Predefined Types

All common SQL column types are represented as subclasses of MappedField. They include:

• MappedBinary
• MappedBoolean
• MappedDate
• MappedDateTime
• MappedDecimal
• MappedDouble
• MappedEnum
• MappedInt
• MappedLong
• MappedString
• MappedText

There are also some more specialized field types:

• MappedCountry
• MappedEmail
• MappedGender
• MappedLocale
• MappedPostalCode

Making Your Own Field Types

// TODO

Validations

Whenever you call validate on a Mapper entity, the framework will validate all the entity’s fields and return a list o FieldErrors (empty if no validation error is found).

Like this:

val person = Person.create.name(“K”)

person.validate match {

  case Nil => S.notice(“Person is valid”)

  case errors:List[FieldError] => S.error(errors)  // S.error will handle this properly

} 

However, we have to tell Mapper how we want our fields to be validated: we can validade a person’s age, email, length of name etc. And to do this we just need to provide a function that receives the field values as a parameter, and returns a list of FieldErrors.

For instance, let’s validate the Person’s name length:

object name extends MappedString(this, 100) {

    def validateNameMinLength(name : String) = {

      if (name.length < 3) {

        List(FieldError(this, “Name too short, dude!”))

      } else {

        List[FieldError]()

      }

    }

  }

And finally we just have to tell the field that we want to use that function for validations. To accomplish this every MappedField has a method called validations which returns a List of validation functions for that field. We just have to append our function to that list by overriding the default method, like this:

object name extends MappedString(this,100) {
  override def validations =  validateNameMinLength _ :: Nil
}

Obs: MappedString and some other String valued fields, mix the trait StringValidators in, which provides some predefined validation functions like: valMinLen, valMaxLen, valRegex. See the Scaladocs for more information.

Filtering

Filters are functions that transform the value before it is set on the Field. These transformation functions are also applied before the value is used in a query.

Some common filters, defined in net.liftweb.util.StringValidators are trim, notNull, toUpper and toLower.

On any MappedField you are able to define Filters by overriding the setFilter method:

object name extends MappedString(this,100) {

  override def setFilter = trim _ :: toUpper _ :: super.setFilter

}

In this example, whenever the name field is set with a value, the value will be trimmed and uppercased.

Relationships

While Mapper is a great way to act a persistent store, relationships are at the heart of relational databases and Mapper accordingly supports them. In most apps you build with Mapper you’ll probably have at least a few situations with there are clear relationships between different models that you want to represent.

OneToMany

Continuing to use our blog post model, blog posts can have many comments, though each comment can only belong to a single blog post. This is a classic one-to-many relationship.

Here’s some code:

package com.mobtest {
package model {

import net.liftweb.mapper._

class Comment extends LongKeyedMapper[Comment] {
  def getSingleton = Comment

  def primaryKeyField = id
  object id extends MappedLongIndex(this)
  object post extends LongMappedMapper(this, Post)
  object author extends MappedString(this, 40)
  object comment extends MappedString(this, 140)
}

object Comment extends Comment with LongKeyedMetaMapper[Comment]

}
}

We can now work with Comments and have them belong to Posts:

val thePost: Post = ...
val newComment: Comment = Comment.create.post(thePost).author("Lift Fanatic").comment("Lift rocks!")
newComment.save

Likewise we can get the Post from the Comment:

val aComment: Comment = ...
val postId: Long = aComment.post
val thePost: Post = aComment.post.obj

Note that we needed to refer to obj to retrieve the actual Post instance. The database query to retrieve the Post is only executed when you reference obj. If you would like the Post to be retrieved when the Comment is fetched from the database, for instance to reduce the total number of database queries, add a PreCache QueryParam to your initial query to ensure an SQL JOIN is done. As always, more information can be found in its Scaladocs.

Of course, a one-to-many relationship goes in two directions and we would like our Posts to know about all their Comments. Here’s an updated version of the Post model:

class Post extends LongKeyedMapper[Post] with OneToMany[Long, Post] {
  def getSingleton = Post
  
  def primaryKeyField = id
  object id extends MappedLongIndex(this)
  object title extends MappedString(this, 140) {
    override def dbIndexed_? = true
    override def defaultValue = "New Blog Post"
  }
  object contents extends MappedText(this)
  object published extends MappedBoolean(this)

  object comments extends MappedOneToMany(Comment, Comment.post, OrderBy(Comment.id, Ascending))
}

You can now use the comments field like a normal collection:

val thePost: Post = ...
val newComment: Comment = Comment.create.author("Lift Fanatic").comment("This is great!")
val currentComments: List[Comment] = post.comments.toList
post.comments -= newComment // no change
post.comments += newComment
post.save
newComment.post == thePost.id

As you can see, comments behaves like a mutable collection. Because we added the new Comment to the Post before we saved the Post, we can subsequently access the post field of the Comment despite never explicitly setting it.

OneToOne

If you’re looking to model a one-to-one relationship, just use a one-to-many relationship. The only potential hassle is that you’ll have a List[B] instead of a Box[B].
You can use MappedLongForeignKey for a OneToOne relationship, but note that it works a little different than a OneToMany.

Things to consider:

You can add a OneToOne relationship by adding an object like this:

object postAuthor extends MappedLongForeignKey(this, PostAuthorInformation)

You don’t have to do anything else on your PostAuthorInformation model class (unless you want bi-directional communication, which I haven’t tried yet).

Creating a related row on PostAuthorInformation

val postAuthorInfo= PostAuthorInformation.create.name(“Diego”)

postAuthorInfo.save

val post= Post.create.postAuthor(postAuthorInfo.id)

post.save

• Note how you need to save the related postAuthorInfo object before you assign it to the postAuthor field name on the Post class
• Also note how you need to use postAuthorInfo.id instead of postAuthorInfo when you assign the related row to the Post class.
• See This thread for ways to avoid the extra save

ManyToMany

Many-to-many relationships work similarly to one-to-many relationships, so we can get up to speed on them in no time flat. Let’s add tags to our blog posts:

package com.mobtest {
package model {

import net.liftweb.mapper._

class Tag extends LongKeyedMapper[Tag] with ManyToMany {
  def getSingleton = Tag

  def primaryKeyField = id
  object id extends MappedLongIndex(this)
  object tag extends MappedString(this, 10)
  object posts extends MappedManyToMany(PostTags, PostTags.tag, PostTags.post, Post)
}

object Tag extends Tag with LongKeyedMetaMapper[Tag]

object PostTags extends PostTags with LongKeyedMetaMapper[PostTags]

class PostTags extends LongKeyedMapper[PostTags] with IdPK {
  def getSingleton = PostTags
  object post extends LongMappedMapper(this, Post)
  object tag extends LongMappedMapper(this, Tag)
}

}
}

Notice how we needed to create an additional PostTags model to represent the join table the database will need to use behind the scenes to keep track of the many-to-many relationships between Tags and Posts. Depending on your background you may be used to your ORM doing this for you but it really isn’t that hard.

Of course, we also need to update our Post model:

class Post extends LongKeyedMapper[Post] with OneToMany[Long, Post] with ManyToMany {
  def getSingleton = Post
  
  def primaryKeyField = id
  object id extends MappedLongIndex(this)
  object title extends MappedString(this, 140) {
    override def dbIndexed_? = true
    override def defaultValue = "New Blog Post"
  }
  object contents extends MappedText(this)
  object published extends MappedBoolean(this)

  object comments extends MappedOneToMany(Comment, Comment.post, OrderBy(Comment.id, Ascending))
  object tags extends MappedManyToMany(PostTags, PostTags.post, PostTags.tag, Tag)
}

Now we can use the posts and tags fields just like Post’s comments field.

Various Helper Traits

IdPK

Remember how we have a MappedLongIndex field and a primaryKeyField method in our Post model? This is because instances of LongKeyedMapper must have, well, a Long field that is the primary key. Very often you’ll be using just such a pattern, so you can simple mixin the IdPK trait instead.

Here’s what the model class looks like now:

class Post extends LongKeyedMapper[Post] with IdPK with OneToMany[Long, Post] with ManyToMany {
  def getSingleton = Post
  
  object title extends MappedString(this, 140) {
    override def dbIndexed_? = true
    override def defaultValue = "New Blog Post"
  }
  object contents extends MappedText(this)
  object published extends MappedBoolean(this)

  object comments extends MappedOneToMany(Comment, Comment.post, OrderBy(Comment.id, Ascending))
  object tags extends MappedManyToMany(PostTags, PostTags.post, PostTags.tag, Tag)
}

Naturally Mapper also supports String primary keys, though your model class and companion object will need to mixin different traits and you’ll need to have a MappedStringIndex field.

MegaProtoUser

MegaProtoUser provides a complete user system with a login and logout functionality, a password retrieval system, and more.

Here is a simple example of a User mode using it:

package net.liftweb.modelexample {
package model {

import net.liftweb.mapper._

class User extends MegaProtoUser[User] {
  def getSingleton = User // companion object
}

object User extends User with MetaMegaProtoUser[User] {
  override val basePath = "user" :: Nil
  override def screenWrap = Full(<lift:surround with="default" at="content"><lift:bind /></lift:surround>)
}

}
}

As you can see, we need to mixin MegaProtoUser into the model class and MetaMegaProtoUser (quite a mouthful!) into the companion object. Our user model now has a bunch of MappedFields including firstName (a MappedString), lastName (MappedString again), email (MappedEmail), and superUser (MappedBoolean). One useful helper method worth knowing about is niceName, which concatenates the firstName and lastName. For apps that aren’t very complicated just using superUser can be a sufficient access control system.

Just a heads up: If your browser is getting errors when trying to access any of the pages created by MegaProtoUser, make sure you’ve overridden screenWrap, returning a Box[Node], in your companion object. The default is Empty, meaning that your browser will get an HTML fragment and not a full document.

The Menu locations defined by MetaMegaProtoUser should be registered within the boot method in Boot.scala like so:

val entries = Menu(Loc("Home", List("index"), "Home")) :: User.sitemap
LiftRules.setSiteMap(SiteMap(entries :_*))

The Logged-In User

You can test whether a user is logged-in anywhere in your application by calling the loggedIn_? method on the companion. Using our earlier example:

import net.liftweb.modelexample.model.User
val loggedIn: Boolean = User.loggedIn_?

You can access the logged-in user anywhere via the currentUser method. This method returns a Box[User], reflecting the fact that a there is not always a user logged in. For example:

import net.liftweb.modelexample.model.User
User.currentUser match {
  case Full(user) => "Hello " + user.niceName + "!"
  case _ => "Who are you? Please login."
}

OpenID

The lift-openid module provides traits that you can mix-in to MegaProtoUser to support OpenID logins. See the OpenID wiki page for more information.

CreatedUpdated

The CreatedUpdated trait, when mixed into the Mapper model class, provides createdAt and updatedAt MappedDateTime fields which are automatically updated with timestamps when appropriate.

Others

There are a variety of other traits you can mix into your models to provided additional functionality. For instance, Cascade will delete the children represented by this field when the parent is deleted. See the Mapper Scaladocs for more information.