![]() ![]() 4JJavaError: An error occurred while calling z. While explicit lists are fine for small cases, I need this to work on many files, so explicit will not work in the long run.įor all the failed cases, here are the follow on steps and the errors: llist = llect()įile "C:/spark/HW1/test2.py", line 30, in įile "c:\spark\python\lib\pyspark.zip\pyspark\rdd.py", line 1197, in collectįile "c:\spark\python\lib\py4j-0.10.9.5-src.zip\py4j\java_gateway.py", line 1321, in _call_įile "c:\spark\python\lib\py4j-0.10.9.5-src.zip\py4j\protocol.py", line 326, in get_return_value In short, explicit file lists work fine, but wildcards or just naming the directory does not. #files = sc.textFile("C:/spark/HW1/data") #files = sc.wholeTextFiles("C:/spark/HW1/data/*") Here are my experiments with what worked and what did not work: #THESE WORK In order to keep my testing really simple, all I am trying to do is load the text files into an RDD and print out the contents. ![]() I am working on Windows 10 with Python 8.5, Java 8, and Anaconda 3. ![]() In spite of that, I still cannot get it to work on my system. I have read multiple tutorials and Q
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |