shapeless
Generic programming for Scala
I've seen an object (probably a function) called "at" sprinkled throughout the shapeless source and in code that uses shapeless. In particular, it is used in the answer to this other question. Here is the code snippet:
object iterateOverHList extends Poly1 {
implicit def iterable[T, L[T] <: Iterable[T]] = at[L[T]](_.iterator)
}
I've had some clue that it is related to the apply method of the ~> type. What specifically does "at" do, and where is it defined?
Source: (StackOverflow)
The following gist has the code for an idea I am playing with
package com.test1
import scala.language.implicitConversions
import shapeless._
import FromTraversable._
import Traversables._
import Nat._
import Tuples._
trait ToArity[P, N <: Nat]
object ToArity {
implicit def prod1[P <: Product1[_]] = new ToArity[P, _1] {}
implicit def prod2[P <: Product2[_, _]] = new ToArity[P, _2] {}
// ad nauseum...
}
trait SizedHListAux[A, N <: Nat, T <: HList]
object SizedHListAux {
implicit def base[A, H <: HList] = new SizedHListAux[A, _0, HNil] {}
implicit def induct[A, H <: HList, N <: Nat, P <: Nat](implicit r: PredAux[N,P], k: SizedHListAux[A, P, H]) = new SizedHListAux[A, N, A :: H] {}
}
trait SomeFun {
type Result
def apply(): Result
}
// I want to abstract over A, the contained type in the List
// over P the Product type which is the arg notably its arity
// This means we need to recover arity of the Product type and render it in value space
// and also means that we need to compute the type of the intermediate HList
object SomeFun {
def produce(m: SomeFun): m.Result = m()
implicit def fromF1[T, A, P <: Product, N <: Nat, H <: HList](f1: (P => T, List[A]))(implicit k: ToArity[P, N], toI: ToInt[N], l: SizedHListAux[A, N, H], toHL: FromTraversable[H], tp: TuplerAux[H, P]) =
new SomeFun {
type Result = (T, List[A])
def apply(): Result = {
val (f, as) = f1
val (ts, rest) = (as.take(toI()), as.drop(toI()))
f((toHL(ts).get).tupled) -> rest
}
}
// Debug Arity checker
def printArity[P <: Product, N <: Nat](p: P)(implicit k: ToArity[P, N], toI: ToInt[N]) = println("Arity: " + toI())
}
object Test {
val thedata = List("foo", "bar", "baz", "bob")
val tfn = (x: (String, String)) => println("%s and %s".format(x._1, x._2))
def foo = SomeFun.printArity("a" -> "b")
//def doit = SomeFun.produce((tfn, thedata)) // Adding this line does not compile
}
The idea is that you use a function's argument arity, in this case the arity of a Product type, to drive parsing of an associated List[A]. Kind of like using sticky tape to peel off layers of graphene from graphite, i.e. the type of the functions pull things out of the list. This is just an sketch using a single contained type, but I imagine it could be generalised. The important facet is that the functions themselves are unaware of the List processing.
However...the concept seems to fail when trying to resolve the ToArity[P,N] implicit. On its own ToArity is resolvable as evidenced by printArity().
Can someone shed some light on why this is not resolvable in the context of fromF1? Is it that it can't resolve all of the dependent implicits and then registers the error with the first, i.e. an N cannot be found to satisfy ToArity, ToInt and SizedHListAux?
Source: (StackOverflow)
I stayed up way too late last night trying to figure out this Shapeless issue and I'm afraid it's going to eat my evening if I don't get it off my chest, so here goes.
In this minimized version I'm just defining a type class that will recursively convert case classes into heterogeneous lists:
import shapeless._
trait DeepHLister[R <: HList] extends DepFn1[R] { type Out <: HList }
trait LowPriorityDeepHLister {
type Aux[R <: HList, Out0 <: HList] = DeepHLister[R] { type Out = Out0 }
implicit def headNotCaseClassDeepHLister[H, T <: HList](implicit
dht: DeepHLister[T]
): Aux[H :: T, H :: dht.Out] = new DeepHLister[H :: T] {
type Out = H :: dht.Out
def apply(r: H :: T) = r.head :: dht(r.tail)
}
}
object DeepHLister extends LowPriorityDeepHLister {
implicit object hnilDeepHLister extends DeepHLister[HNil] {
type Out = HNil
def apply(r: HNil) = HNil
}
implicit def headCaseClassDeepHLister[H, R <: HList, T <: HList](implicit
gen: Generic.Aux[H, R],
dhh: DeepHLister[R],
dht: DeepHLister[T]
): Aux[H :: T, dhh.Out :: dht.Out] = new DeepHLister[H :: T] {
type Out = dhh.Out :: dht.Out
def apply(r: H :: T) = dhh(gen.to(r.head)) :: dht(r.tail)
}
def apply[R <: HList](implicit dh: DeepHLister[R]): Aux[R, dh.Out] = dh
}
Let's try it out! First we need some case classes:
case class A(x: Int, y: String)
case class B(x: A, y: A)
case class C(b: B, a: A)
case class D(a: A, b: B)
And then (note that I've cleaned up the type syntax for the sake of this not being a totally unreadable mess):
scala> DeepHLister[A :: HNil]
res0: DeepHLister[A :: HNil]{
type Out = (Int :: String :: HNil) :: HNil
} = DeepHLister$$anon$2@634bf0bf
scala> DeepHLister[B :: HNil]
res1: DeepHLister[B :: HNil] {
type Out = (
(Int :: String :: HNil) :: (Int :: String :: HNil) :: HNil
) :: HNil
} = DeepHLister$$anon$2@69d6b3e1
scala> DeepHLister[C :: HNil]
res2: DeepHLister[C :: HNil] {
type Out = (
((Int :: String :: HNil) :: (Int :: String :: HNil) :: HNil) ::
(Int :: String :: HNil) ::
HNil
) :: HNil
} = DeepHLister$$anon$2@4d062faa
So far so good. But then:
scala> DeepHLister[D :: HNil]
res3: DeepHLister[D :: HNil] {
type Out = ((Int :: String :: HNil) :: B :: HNil) :: HNil
} = DeepHLister$$anon$2@5b2ab49a
The B didn't get converted. If we turn on -Xlog-implicits this is the last message:
<console>:25: this.DeepHLister.headCaseClassDeepHLister is not a valid implicit value for DeepHLister[shapeless.::[B,shapeless.HNil]] because:
hasMatchingSymbol reported error: diverging implicit expansion for type DeepHLister[this.Repr]
starting with method headNotCaseClassDeepHLister in trait LowPriorityDeepHLister
DeepHLister[D :: HNil]
^
Which doesn't make sense to me—headCaseClassDeepHLister should be able to generate DeepHLister[B :: HNil] just fine, and it does if you ask it directly.
This happens on both 2.10.4 and 2.11.2, and with both the 2.0.0 release and master. I'm pretty sure this has to be a bug, but I'm not ruling out the possibility that I'm doing something wrong. Has anyone seen anything like this before? Is there something wrong with my logic or some restriction on Generic I'm missing?
Okay, thanks for listening—maybe now I can go read a book or something.
Source: (StackOverflow)
I was trying to do some stuff last night around accepting and calling a generic function (i.e. the type is known at the call site, but potentially varies across call sites, so the definition should be generic across arities).
For example, suppose I have a function f: (A, B, C, ...) => Z. (There are actually many such fs, which I do not know in advance, and so I cannot fix the types nor count of A, B, C, ..., Z.)
I'm trying to achieve the following.
How do I call f generically with an instance of (A, B, C, ...)? If the signature of f were known in advance, then I could do something involving Function.tupled f or equivalent.
How do I define another function or method (for example, some object's apply method) with the same signature as f? That is to say, how do I define a g for which g(a, b, c, ...) type checks if and only if f(a, b, c, ...) type checks? I was looking into Shapeless's HList for this. From what I can tell so far, HList at least solves the "representing an arbitrary arity args list" issue, and also, Shapeless would solve the conversion to and from tuple issue. However, I'm still not sure I understand how this would fit in with a function of generic arity, if at all.
How do I define another function or method with a related type signature to f? The biggest example that comes to mind now is some h: (A, B, C, ...) => SomeErrorThing[Z] \/ Z.
I remember watching a conference presentation on Shapeless some time ago. While the presenter did not explicitly demonstrate these things, what they did demonstrate (various techniques around abstracting/genericizing tuples vs HLists) would lead me to believe that similar things as the above are possible with the same tools.
Thanks in advance!
Source: (StackOverflow)
I am new to shapeless and have been trying to practice some type level programming. I took Problem #1 from Project Euler as my first challenge.
I started by writing regular scala code:
object ProjectEuler1 {
def e1(limit: Int) = (1 until limit).foldLeft(0) {
case (acc, x) if x % 3 * x % 5 == 0 => acc + x
case (acc, _) => acc
}
val out = e1(10)
assert(out == 23)
}
Then, I came up with this working shapeless implementation using poly:
object ProjectEuler1Shapeless extends App {
import shapeless._
import nat._
import ops.nat._
import poly._
import test.typed
trait eLP extends Poly1 {
implicit def default[A <: Nat] = at[A] { _ => _0 }
}
object e extends eLP {
implicit def match3[A <: Nat](implicit ev: Mod.Aux[A, _3, _0]) = at[A](identity)
implicit def match5[A <: Nat](implicit ev: Mod.Aux[A, _5, _0]) = at[A](identity)
}
object sum extends Poly2 {
implicit def sum[A <: Nat, B <: Nat, Z <: Nat](implicit s: Sum.Aux[A, B, Z],
z: Witness.Aux[Z]) =
at[A, B] { (_, _) => z.value }
}
type _23 = Succ[_22]
val l = _1 :: _2 :: _3 :: _4 :: _5 :: _6 :: _7 :: _8 :: _9 :: HNil
val out = l.map(e).foldLeft(_0)(sum)
typed[_23](out)
}
Next, I wanted to change the function so that I don't need to manually create a list. Instead it accepts a "limit" as an argument like the regular scala code. I came up with this:
object ProjectEuler1Shapeless2 extends App {
import shapeless._
import nat._
import ops.nat._
import test.typed
class E1[I <: Nat, N <: Nat]
trait ELP0 {
implicit def default[I <: Nat, M <: Nat] = new E1[I, _0]
}
trait ELP1 extends E1LP0 {
implicit def match3[A <: Nat](implicit ev: Mod.Aux[A, _3, _0]) = new E1[A, A]
implicit def match5[A <: Nat](implicit ev: Mod.Aux[A, _5, _0]) = new E1[A, A]
}
object E1 extends E1LP1 {
implicit def combine[I <: Nat, L <: Nat, M <: Nat](implicit e1: E1[I, L],
m: E1[Succ[I], M],
sum: Sum[L, M]) =
new E1[Succ[Succ[I]], sum.Out]
}
def e1[N <: Nat](limit: Nat)(implicit e: E1[limit.N, N], w: Witness.Aux[N]): N = w.value
val f1 = e1(1)
typed[_0](f1)
val f2 = e1(2)
typed[_0](f2)
val f3 = e1(3)
typed[_3](f3) // Does not compile!
}
I've gotten stuck here. The compiler is telling me it found _0. I guess it's picking up the instance from def default.
Any tips on how I can fix this? I have a feeling my strategy for solving this problem might be a little weird also. Any pointers on how I can make this shapeless code more idiomatic are greatly appreciated.
My original strategy was to create a hylomorphism. I noticed there is an unfold example in the shapeless git but its complexity escapes me at the moment.
Source: (StackOverflow)
I need a Map where I put different types of values (Double, String, Int,...) in it, key can be String.
Is there a way to do this, so that I get the correct type with map.apply(k) like
val map: Map[String, SomeType] = Map()
val d: Double = map.apply("double")
val str: String = map.apply("string")
I already tried it with a generic type
class Container[T](element: T) {
def get: T = element
}
val d: Container[Double] = new Container(4.0)
val str: Container[String] = new Container("string")
val m: Map[String, Container] = Map("double" -> d, "string" -> str)
but it's not possible since Container takes an parameter. Is there any solution to this?
Source: (StackOverflow)
I am trying to learn Shapeless (using version 2.10.2). I have created a very simple extensible record:
val rec1 = ("foo" ->> 42) :: HNil
According to the REPL, this has type
shapeless.::[Int with shapeless.record.KeyTag[String("foo"),Int],shapeless.HNil]
I am trying to define a simple function:
def fun(x: ::[Int with KeyTag[String("foo"), Int], HNil]) = x("foo")
but it does not even compile. I cannot use a String("foo") in the type declaration, and get an error.
I have two questions:
- How can I specify the type of the extensible record in my code?
- When working with records with more fields, the length and complexity of the type declaration will be unmanageable. Is there a way to create an alias for the type, given a particular instance of a record, or some other workaround?
EDIT
I have found that:
val rec1 = ("foo" ->> 42) :: HNil
val rec2 = ("foo" ->> 43) :: HNil
var x = rec1
x = rec2
works well. I conclude rec1, rec2, and x are of the same type. I just don't know how to express that type in code!
Source: (StackOverflow)
I'm thinking of doing something similar to Safely copying fields between case classes of different types but with reordered fields, i.e.
case class A(foo: Int, bar: Int)
case class B(bar: Int, foo: Int)
And I'd like to have something to turn a A(3, 4) into a B(4, 3) - shapeless' LabelledGeneric comes to mind, however
LabelledGeneric[B].from(LabelledGeneric[A].to(A(12, 13)))
results in
<console>:15: error: type mismatch;
found : shapeless.::[shapeless.record.FieldType[shapeless.tag.@@[Symbol,String("foo")],Int],shapeless.::[shapeless.record.FieldType[shapeless.tag.@@[Symbol,String("bar")],Int],shapeless.HNil]]
(which expands to) shapeless.::[Int with shapeless.record.KeyTag[Symbol with shapeless.tag.Tagged[String("foo")],Int],shapeless.::[Int with shapeless.record.KeyTag[Symbol with shapeless.tag.Tagged[String("bar")],Int],shapeless.HNil]]
required: shapeless.::[shapeless.record.FieldType[shapeless.tag.@@[Symbol,String("bar")],Int],shapeless.::[shapeless.record.FieldType[shapeless.tag.@@[Symbol,String("foo")],Int],shapeless.HNil]]
(which expands to) shapeless.::[Int with shapeless.record.KeyTag[Symbol with shapeless.tag.Tagged[String("bar")],Int],shapeless.::[Int with shapeless.record.KeyTag[Symbol with shapeless.tag.Tagged[String("foo")],Int],shapeless.HNil]]
LabelledGeneric[B].from(LabelledGeneric[A].to(A(12, 13)))
^
How do I reorder the fields in the record (?) so this can work with a minimum of boilerplate?
Source: (StackOverflow)
I'm trying to learn shapeless (2.0.0). It seems like an amazing tool and I'm very excited about it, but I am having problems moving forward. Because there is not yet much documentation, I've been poring over examples and the source code. I am having difficulties because most examples use multiple shapeless concepts and, in the source code, one shapeless type will often make use of others, so I end up going down the shapeless rabbit hole, so to speak. I think it would be helpful to have a list of the important features of the shapeless API along with a simple description of what each one does. As I'm clearly unqualified to make such a list, I am asking you, the humans of Stack Overflow!
For each feature, please include as much as you can of the following:
The feature's name and how to import it.
A short, simple description of what it does.
Why is this feature important / why would someone bother to use it?
A simple example that uses as few other shapeless or advanced Scala concepts as possible.
By a feature of the API, I mean a single thing (e.g., a type, a function, an object, etc.), or small set of closely coupled such things, that is defined by shapeless 2.0 and can be imported and used in a program. I am not referring to general concepts such as higher order polymorphism or type-level recursion. And please only include one feature per answer. Maybe if there are enough answers and enough others also use this list, we can use the votes on the answers to rank the importance of the different features.
Note: I am aware of this feature list. I think it's great, and it has helped me a lot. However, I'm looking for something more similar to API documentation than a list of things you can do. I can understand many of the examples and infer the purposes of some features from them, but I will often get tripped up on some particular piece and be unable to figure out its function.
Source: (StackOverflow)
I have done a few implementations of HList now. One based on Daniel Spiewak's High Wizardry in the Land of Scala talk and another based on a post in Apocalisp blog. The goal was to have a heterogenous list of which is not heterogenous in the primary type but rather the higher kind. For example:
val requests = Request[String] :: Request[Int] :: HNil
I would be able to do a map across the list to perform the request and result in a heterogenous list of the higher kind. So:
requests.map(execute)
should equal
String :: Int :: HNil
Sadly all my attempts have resulted in an HList of Any. Here is the code from a recent attempt:
class Request[+Out](o:Out) {
type O = Out
def v:O = o
}
object HList {
trait Func[-Elem,Out] {
type Apply[E <: Elem] <: Out
def apply[N <: Elem](e:N):Apply[N]
}
sealed trait HList[Base] {
type Head <: Base
type Tail <: HList[Base]
type Map[Out,F <: Func[Base,Out]] <: HList[Out]
def head:Head
def tail:Tail
def ::[A <: Base](a:A):HList[Base]
def map[Out,F <: Func[Base,Out]](f:F):Map[Out,F]
}
case class HNil[Base]() extends HList[Base] {
type Head = Nothing
type Tail = Nothing
type Map[Out,F <: Func[Base,Out]] = HNil[Out]
def head = error("Head of an empty HList")
def tail = error("Head of an empty HList")
def ::[A <: Base](a:A) = HCons(a,this)
def map[Out,F <: Func[Base,Out]](f:F) = new HNil[Out]
}
case class HCons[Base,A <: Base,B <: HList[Base]](head: A, tail: B) extends HList[Base] {
type Head = A
type Tail = B
type Map[Out,F <: Func[Base,Out]] = HCons[Out,F#Apply[Head],Tail#Map[Out,F]]
def ::[C <: Base](c:C) = HCons(c,this)
def map[Out,F <: Func[Base,Out]](f:F) =
HCons(f(head),tail.map(f))
}
val :: = HCons
}
object Test extends Application {
import HList._
val HNil = new HNil[Request[_]]
val list = new Request[Int](1) :: new Request[String]("1") :: HNil
val (a :: b :: HNil) = list
val y:Request[String] = b
val results = list.map[Any,Unwrap.type](Unwrap)
val i:Int = results.head
}
import HList._
object Unwrap extends Func[Request[Any],Any] {
type Apply[I <: Request[Any]] = I#O
def apply[N <: Request[Any]](e:N) = null.asInstanceOf[Apply[N]]
}
The other attempt was based on the Apocalisp version which uses fold to create a new HList and again it resulted in a HList of Any types. Any tips would be appreciated.
Source: (StackOverflow)
The following code is Idris:
natAssociative : (a : Nat) -> (b : Nat) -> (c : Nat) -> (a + b) + c = a + (b + c)
natAssociative Z b c = the (b + c = b + c) refl
natAssociative (S k) b c = replace {P=\x => S (k + b) + c = S x} (natAssociative k b c) refl
I'm having a very tough time translating that to shapeless. I've tried a few different encodings, but I think this was the most promising start:
import scalaz.Leibniz._
import shapeless.{ HNil, Nat, Succ, Poly3 }
import shapeless.Nat._
import shapeless.ops.nat._
object natAssociative extends Poly3 {
implicit def case0[B <: Nat, C <: Nat]: Case[_0, B, C] = at[_0, B, C] {
case (Nat._0, b, c) => refl[Sum[B, C]#Out]
}
implicit def caseSucc[K <: Nat, B <: Nat, C <: Nat] = ???
}
I'm having trouble with induction and making Scala recognise that we have 2 possible cases to recurse to. Is there a trick for encoding this part?
Source: (StackOverflow)
Given a Shapeless HList where every list element shares the same type constructor, how can the HList be sequenced?
For example:
def some[A](a: A): Option[A] = Some(a)
def none[A]: Option[A] = None
val x = some(1) :: some("test") :: some(true) :: HNil
val y = sequence(x) // should be some(1 :: "test" :: true :: HNil)
def sequence[L <: HList : *->*[Option]#λ, M <: HList](l: L): Option[M] =
???
I tried to implement sequence like this:
object optionFolder extends Poly2 {
implicit def caseOptionValueHList[A, B <: HList] = at[Option[A], Option[B]] { (a, b) =>
for { aa <- a; bb <- b } yield aa :: bb
}
}
def sequence[L <: HList : *->*[Option]#λ, M <: HList](l: L): Option[M] = {
l.foldRight(some(HNil))(optionFolder)
}
But that does not compile:
could not find implicit value for parameter folder: shapeless.RightFolder[L,Option[shapeless.HNil.type],SequencingHList.optionFolder.type]
Any tips on implementing this for either a specific example like Option or for an arbitrary Applicative?
Source: (StackOverflow)
Can someone explain to me in simple terms what the Shapeless library is for?
Scala has generics and inheritance functionality so I'm a bit confused what Shapeless is for.
Maybe a use case to clarify things would be helpful.
Source: (StackOverflow)
I'm using shapeless in Scala, and I'd like to write a function allPairs that will take two HLists and return an HList of all pairs of elements. For example:
import shapeless._
val list1 = 1 :: "one" :: HNil
val list2 = 2 :: "two" :: HNil
// Has value (1, 2) :: (1, "two") :: ("one", 2) :: ("one", "two") :: HNil
val list3 = allPairs(list1, list2)
Any idea how to do this?
Also, I'd like to emphasize I'm looking for a function, not an inlined block of code.
Source: (StackOverflow)
Having
(Some(1), Some(2))
I expect to get
Some((1, 2))
and having
(Some(1), None)
I expect to get
None
Source: (StackOverflow)