current community

your communities

more stack exchange communities

company blog
Stack Exchange Inbox Reputation and Badges
tour help

Tagged Questions

Scala is a general purpose programming language principally targeting the Java Virtual Machine. Designed to express common programming patterns in a concise, elegant, and type-safe way, it fuses both imperative and functional programming styles. Its key features are: advanced static type system ...

learn more… | top users | synonyms | scala jobs

0
votes
0answers
7 views

Using getter method in Spark RDD

I got simple getter method's unexpected behavior in Spark. 1) I have an abstract class, with a variable and its getter. abstract class AbstractObject { private var _date: String = _ def ...
0
votes
0answers
7 views

Read new s3 file paths from spark streaming

I'd like to use spark streaming to monitor a s3 directory and return the path of any new files that are added to that directory. Neither textFileStream nor fileStream seem to be able to do this. Is ...
0
votes
0answers
3 views

Scala Parallel Print Hanging the console

I am new to this Scala world and I am trying some exercises from a book. So, I have an example that print a vector in sequential and parallel fashion. The former works perfectly and the later hangs ...
1
vote
0answers
8 views

Use Enumeration inside of Regex pattern matching

I have written this code object Foo extends Enumeration { val X = Value("X") val Y = Value("Y") } val regex = """([A-Z]+)(\d+)""".r val input = "ABC123" input match { case regex(x, num) if x =...
0
votes
0answers
7 views

Splicing json text into json4s ast without parsing

I have a string, which contains valid json. I'd like to add that json as a JField in a larger json4s AST I'm building, but it'd be nice to avoid having to parse the json into an AST first. I trust it....
2
votes
1answer
94 views

My API is all returning Future[Option[T]], how to combine them nicely in a for-compr

All of my API methods return Future[Option[T]], trying to figure out how to elegantly perform the following: case class UserProfile(user: User, location: Location, addresses: Address) The below code ...
0
votes
0answers
7 views

Spark JobServer JobEnvironment

def main(args: Array[String]) { val conf = new SparkConf().setMaster("local[4]").setAppName("LongPiJob") val sc = new SparkContext(conf) val env = new JobEnvironment { def jobId: String = "...
-4
votes
0answers
15 views

Generate random text with Spark

Hello i want to generate random line from a set of words array with spark and write it to a file. This is my code can someone explain me how to continue? Thanks object RandomText { var selectedLines ...
-1
votes
0answers
18 views

Spark: Scala program for HBase bulk loading

Suppose I have a file with following format: SubjectName ColumnName Value s1 c1 v1 s1 c2 v2 s1 c3 v3 s2 c1 v4 s2 ...
0
votes
0answers
18 views

Play Framework: showing image via html

So, I'm new to Play and I encounter this problem: I have a database (using jpa), let's say "Movies" and each movie has a name and an imgName. The images are stored in public-> images folder of the ...
1
vote
0answers
15 views

DI or Service Locator : Injecting implementations at run-time ( no static binding ) in scala

i have a use case where i would like to offer a simple API to extend the functionality of my scala application. i've spent the last couple of days trying to find a java/scala DI framework or library ...
1
vote
2answers
49 views

How to create a new list inside foreach in scala

I am newbie to scala and just trying out stuff, below is what I am trying scala> var col = List[String]() col: List[String] = List() scala> List("a", "b", "c").foreach(x => x :: col) ...
0
votes
0answers
7 views

how to parse the wiki infobox json with scala spark

I was trying to get the data from json data which I got it from wiki api https://en.wikipedia.org/w/api.php?action=query&prop=revisions&rvprop=content&format=json&titles=Rajanna&...
-4
votes
1answer
20 views

how to use mongodb with play, scala and intellij [on hold]

I want to create a scala based web application using MongoDB as database in Play. How can I do this? What support does Play provide to use Mongodb?
0
votes
0answers
15 views

Setting up a linear regression in spark [duplicate]

I am trying to setup a linear regression in spark as per official documentation. My question relates to setLabelCol and setFeaturesCol. From the API, setFeaturesCol() takes a string as a parameter. ...
0
votes
2answers
11 views

How to create sources from resources (or other project files) in SBT

I would like to generate sources from files which are part of the project (I have currently placed them in a resource directory, but this is not a requirement). This is my attempt on it: ...
0
votes
0answers
21 views

Spark scala code for HBase in intellij-IDEA

I added the following jars as external libraries in IntelliJ-IDEA to run simple scala code. But it is saying ZooKeeper exists failed after 4 attempts, ConnectionLossException: KeeperErrorCode = ...
0
votes
0answers
24 views

Does this FlowShape make sense?

I need to Read data that come from TCP client thru port XXXX Process de data Send the same data to several clients connected to port YYYY My approach is to read the data and send it to an actor to ...
-3
votes
1answer
37 views

Scala beginner for-comprehensions and functions

Hey guys i am new to scala and i dont knowe what i am doing wrrong to get the right typ into the yield statement: def prim(n:Int): (Boolean, List[Int]) = divsers(n) -> List(1,n) //flatMap for-...
-2
votes
0answers
12 views

Dataframe to BufferReader, how to Conver?

I am trying to use Spark Custom Stream Receiver to receive RDBMS table as stream. In receiver, I got table as Dataframe, now I need to convert that to BufferedReaderto store in sparkjob. How can I ...
-2
votes
2answers
28 views

Scala procedure and function differences

I am learning Scala and running below code .I knew functions, that do not return anything is procedures in Scala but when running below code why extra () is coming in output. Here in procedure i am ...
0
votes
3answers
42 views

separate scala list based on matching pattern

i have a list of the following scala trait. How can i separate the list into two, one containing only ValidatedSbcCommand objects and other only containing FailedValidationSbcCommand objects? ...
0
votes
0answers
14 views

Passing case class(Events) to Akka Stream's Source()

I am trying to implement a CQRS solution. After I perform some operation(execute a Command) I pass my generated events which are case classes(ex: AccountCreated) to another actor which then uses akka ...
0
votes
1answer
15 views

Filtering out elements in an RDD with fewest entries

I have an RDD of products (productID, name) and another RDD of people's ratings of those products using the MLLib Rating object (ratingTimestamp, Rating(userID, productID, rating)). Before I do any ...
0
votes
0answers
27 views

Play 2.5 - Run a Java method at certain times of day (cron)

I'm working on a Play 2.5 app that needs to run a method at midday, 2pm, 4pm every day autonomously. So far I have followed another answer on here which has got me most of the way. The application....
0
votes
1answer
31 views

Join on two RDDs using Scala in Spark

I am trying to implement Local Outlier Factor on Spark. So I have a set of points that I read from a file and then for each point find the N nearest neighbors. Each point has an index given to it ...
0
votes
0answers
12 views

How to set filter bases on particular field in json(Jackson framework)?

I have written a Scala code where i'm trying to read json file and able to read json file successfully.Here i want to put filter on particular field(recordTypeCode) of json string, if recordTypeCode ...
0
votes
2answers
71 views

'tee' operation on Scala's option type?

Is there some sort of 'tee' operation on Option in Scala's standard library available? The best I could find is foreach, however its return type is Unit, therefore it cannot be chained. This is what ...
0
votes
2answers
45 views

Type Translation Issues

I have this below Class hierarchy in Java Class Container { private List<? extends Element> element; public List<? extends Element> getElements() { return elements; ...
0
votes
0answers
7 views

java.lang.UnsatisfiedLinkError: no jhdf5 in java.library.path

I get the above error while running a Scala Saddle 1.3.4 application that saves and loads h5 files. I have already tried without success configuring the following (Win64): -Djava.library.path=C:\Dev\...
0
votes
1answer
36 views

Event Sourcing with Scala and Immutable Objects: Is it efficient?

I'm experimenting with Event Sourcing with Scala (I'm new both in this two fields). I'd like to keep everything as immutable as possible, including aggregate roots. As a base, I follow Greg Young's ...
1
vote
1answer
31 views

How to reduce shuffling and time taken by Spark while making a map of items?

I am using spark to read a csv file like this : x, y, z x, y x x, y, c, f x, z I want to make a map of items vs their count. This is the code I wrote : private def genItemMap[Item: ClassTag](data: ...
0
votes
0answers
34 views

Scala map and Futures - Call by name? [duplicate]

This is a clarification on the topic of call by name future's map internally takes def map[S](f : scala.Function1[T, S])(implicit executor..... future's apply method takes def apply[T](body : => T)...
1
vote
1answer
66 views

Is map() call by name?

Is map working by call by name? The function which I pass to a map is executed over the elements in the sequence. Since it is being evaluated not during invocation, I believe it is call by name. But I ...
0
votes
1answer
29 views

send scheduled message to self within future

I just saw this code somewhere : class FooActor extends Actor { def receive = { case FooMsg => methodThatReturnsFuture().map { response => context.system.scheduler.scheduleOnce(1 minutes,...
-2
votes
1answer
11 views

Where is json4s Scala documentation?

I cannot find documentation for json4s Scala library. I've found https://github.com/json4s/json4s or http://json4s.org/ but it's not exhaustive doc. I would expect complete documentation like http://...
0
votes
0answers
29 views

NewLine character in CSV spark scala is not working

Sample input CSV data defid|defectDescription|actualRootcause (3659,|"User is able to perform Unpend action Note:workitem", |Code) (3660,| Unknown Issue | Code) Sample output CSV data defid|...
0
votes
1answer
51 views

The best way to convert Success into Failure depending on the value of Success

I use Play! Framework with ReactiveMongo plugin (versions 2.3 and 0.11 accordingly). And I have the next code which removes items from MongoDB collection and returns a number of affected items: /** ...
0
votes
0answers
27 views

java.lang.NoClassDefFoundError: org/apache/spark/streaming/api/java/JavaStreamingContext [duplicate]

code : final Session session = connector.openSession(); final PreparedStatement prepared = session.prepare("INSERT INTO spark_test5.messages JSON?"); JavaStreamingContext ssc = new ...
-1
votes
2answers
37 views

What is the most accurate way to convert org.joda.time.LocalDateTime to java.time.LocalDateTime in scala?

I have output of a library function that returns a random org.joda.time.LocalDateTime in the future that needs to be converted to java.time.LocalDateTime. What is the most efficient way to do this ...
0
votes
1answer
28 views

how to extract the column name and data type from nested struct type in spark

How to extract the column name and data type from nested struct type in spark schema getting like this: (events,StructType( StructField(beaconType,StringType,true), StructField(...
0
votes
0answers
26 views

Eclipse to Remote Hadoop Server

I need to write Scala programs from my Eclipse IDE installed on my local desktop to a remote Hadoop server on AWS. My question is, is it possible to connect my Eclipse IDE directly to the Hadoop ...
0
votes
2answers
20 views

Loading JSON file similar to XML in scala

I'm refactoring a system (written in scala) which was earlier using XML files to one using JSON files. In a test case, an XML file is read as follows import scala.xml.XML val xml = XML.loadFile("/...
0
votes
1answer
23 views

how to apply pattern matching on the file name using spark(scala)

I am monitoring a directory in hdfs and if a file is put into it i want to retrieve the name of the file entering the directory and apply pattern matching so as to sort them based on names so far i ...
2
votes
1answer
52 views

Scala – Make implicit value classes available in another scope

I have a package foo which contains class FStream. The package object of foo defines a few implicit value classes that provide extender methods for FStream. I would like to move these value classes ...
0
votes
1answer
14 views

intellij error while importing SBT project

I recently installed IntelliJ on Ubuntu 16.04, but I'm getting this error while trying to start a Scala project: This is my first linux computer, and I installed the oracle 8 jdk through the package ...
0
votes
0answers
20 views

File upload to play/scala server from React/JS results in 400 Bad Request

Getting 400 Bad Request for the following file upload code: React/JS code: uploadFile (e) { var url = "/upload-file"; var fd = new FormData(); fd.append('image', this.state.input.files[...
4
votes
2answers
61 views

Scala currying example

I am new to scala and was going through the coursera course for scala functional development. Below the code snippet explains about currying import math.abs object exercise{ val tolerance = 0....
0
votes
1answer
29 views

How to do filter on Seq[Array[String]] in Scala?

I am trying to get the number files exists in the s3 bucket. I have list of paths as Seq which I am trying to check. I am trying to filter the paths and count but keep getting error. import java.net....
0
votes
2answers
41 views

How to get substring in scala?

How do I get the substring for the string : "s3n://bucket/test/files/*/*" I would like to get s3n://bucket/test/files alone. I tried the split : "s3n://bucket/test/files/*/*".split("/*/*") but ...

1 2 3 4 5 … 1244