Welcome toVigges Developer Community-Open, Learning,Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
987 views
in Technique[技术] by (71.8m points)

write a apache crunch Pcollection to multiple output files

I have a crunch dofn which generates a Pcollection currently i m writing the pcollection to a single avro file i want to write the Pcollection to multiple files.


 PCollection<String> generatedResults = results.parallelDo(new AvroGeneratorDofn(count),Avros.specifics(String.class));
    //generatedResults.write(To.avroFile(outputPath));
    pipeline.write(generatedResults,new AvroFileTarget(outputPath), Target.WriteMode.APPEND);

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Answer

0 votes
by (71.8m points)
等待大神答复

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome to Vigges Developer Community for programmer and developer-Open, Learning and Share
...