Json rest msgs through JsonPost have charset encoding issues
Have had some trouble receiving json rest messages correctly which transforms my nice national special characters to garbage.
The Req.json method will be read with the default encoding and it does not change even if I set the content type charset header. I'm not entirely sure what charset should be default when the charset header is not specified. On one hand it's alright to be able to specify it (even though it is set for the whole vm). On the other hand the json rfc 4627 specifies that utf-8 is the default encoding. I use my own JsonPost-rewrite that uses the latter fallback. It looks like this now:
def charset(req: Req) = {
val props = req.contentType.openOr("").split(";").map(_.split("=")).filter(_.length == 2).map(a => (a(0).trim.toLowerCase, a(1).trim)).toMap
props.get("charset").getOrElse(org.apache.http.protocol.HTTP.UTF_8)
}
// Copied from Req and modified
def json(req: Req): Box[JsonAST.JValue] =
try {
import root.java.io._
req.body.map(b => JsonParser.parse(new InputStreamReader(new ByteArrayInputStream(b), charset(req))))
} catch {
case e: Exception => Failure(e.getMessage, Full(e), Empty)
}
// Copied from RestHelper (JsonBody/JsonPost) and modified
protected trait RealJsonBody {
def body(r: Req): Box[JValue] = json(r)
}
protected lazy val RealJsonPost = new TestPost[JValue] with JsonTest with RealJsonBody
// My code
serve {
case RealJsonPost("rest" :: "memberships" :: Nil, (json, _)) => createMembership(json)
}
Thanks
ThorÅge
The Req.json method will be read with the default encoding and it does not change even if I set the content type charset header. I'm not entirely sure what charset should be default when the charset header is not specified. On one hand it's alright to be able to specify it (even though it is set for the whole vm). On the other hand the json rfc 4627 specifies that utf-8 is the default encoding. I use my own JsonPost-rewrite that uses the latter fallback. It looks like this now:
def charset(req: Req) = {
val props = req.contentType.openOr("").split(";").map(_.split("=")).filter(_.length == 2).map(a => (a(0).trim.toLowerCase, a(1).trim)).toMap
props.get("charset").getOrElse(org.apache.http.protocol.HTTP.UTF_8)
}
// Copied from Req and modified
def json(req: Req): Box[JsonAST.JValue] =
try {
import root.java.io._
req.body.map(b => JsonParser.parse(new InputStreamReader(new ByteArrayInputStream(b), charset(req))))
} catch {
case e: Exception => Failure(e.getMessage, Full(e), Empty)
}
// Copied from RestHelper (JsonBody/JsonPost) and modified
protected trait RealJsonBody {
def body(r: Req): Box[JValue] = json(r)
}
protected lazy val RealJsonPost = new TestPost[JValue] with JsonTest with RealJsonBody
// My code
serve {
case RealJsonPost("rest" :: "memberships" :: Nil, (json, _)) => createMembership(json)
}
Thanks
ThorÅge
Leave a comment