Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

如何支持spark2.3 #80

Open
guoyuhaoaaa opened this issue Aug 29, 2018 · 7 comments
Open

如何支持spark2.3 #80

guoyuhaoaaa opened this issue Aug 29, 2018 · 7 comments

Comments

@guoyuhaoaaa
Copy link

你好,如果我想使用spark2.3,需要对项目做哪些调整??

@guoyuhaoaaa
Copy link
Author

@sinllychen
你好,如果我想使用spark2.3,需要对项目做哪些调整??是需要重写modules下面的spark.jar文件吗?

@sinllychen
Copy link
Contributor

@guoyuhaoaaa 您好,对于EML搭建包来说,需要换掉spark的版本。别的应该不需要替换;对于我们给的示例里面,只需要重写下ETL那部分就好了。ETL那部分用到有一个类在2.3版本里被别的类替换了。别的算子可以直接使用。

@guoyuhaoaaa
Copy link
Author

guoyuhaoaaa commented Aug 30, 2018

@sinllychen
我想问下modules底下那些模块实例,我看都是spark.jar的形式,我想自己添加一些算法模块,这些spark.jar可以提供源码吗??

@sinllychen
Copy link
Contributor

@guoyuhaoaaa 您好,目前我们的算法源码没有开源,抱歉了。

@guoyuhaoaaa
Copy link
Author

guoyuhaoaaa commented Aug 30, 2018

@sinllychen
您好,我是北邮在校的学生,对这一块比较感兴趣。我不求全部开源,您只开源一个模块也可以,比方说就是GBDT_Train也行,就是想学习学习这种模块接口是怎么写的,绝不商用。
我个人邮箱:guo_yu_hao_1993@163.com

@sinllychen
Copy link
Contributor

sinllychen commented Sep 20, 2018

@guoyuhaoaaa 您好,如果您是想学习如何向EML上上传算法的话,您可以参考我们wiki上的这个示例https://github.com/ICT-BDA/EasyML/wiki/Simple-Examples-of-EasyML。
如果是想学习下算法您可以去看看GBDT的原理哈。

@xiangguanlan
Copy link

@sinllychen 您好,ETL的源码能开放一些吗

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants