Commit graph

76 commits

Author SHA1 Message Date
Xiangyu Tian
94f08edbb3 [PPML] Refactor BigDL Attestation Service Deployment of Docker and K8s (#8130)
Refactor BigDL Attestation Service Deployment of Docker image, which split to base image and custom(reference) image.
Update version to 2.4.0-SNAPSHOT.
Refine documents.
2023-04-26 14:28:00 +08:00
Yao Li
d833a765fe update manually build (#8129) 2023-04-24 16:08:56 +08:00
Yao Li
8719674c92 update manually_build.yml (#8126) 2023-04-24 15:29:24 +08:00
Le-Zheng
8b0876f238 add tdx image in action (#8125)
* add tdx image in action

* update

* Update manually_build.yml

* update Readme
2023-04-24 14:37:14 +08:00
Yao Li
f30215a77e delete outdated bigdl kms (#8115) 2023-04-23 15:45:55 +08:00
Yao Li
981bded4b3 fix bigdl-kms-reference (#8110) 2023-04-23 14:08:53 +08:00
Yao Li
c7630f759a [PPML] Add bigdl-kms into manually build (#8105)
* update readme

* update manually_build.yml

* Update manually_build.yml

* fix format

* udpate

* update
2023-04-23 11:51:00 +08:00
Wang Jian
96c9343ef2 [PPML] Update occlum production image build dir (#8098) 2023-04-21 11:46:42 +08:00
Heyang Sun
814f5bd915 add no_proxy for bigdl-kms (#7996) 2023-04-06 09:18:53 +08:00
Shaojie Cui
f885466475 [CICD]fix: typo in build (#7990) 2023-04-04 14:30:44 +08:00
Shaojie Cui
8a24ae76a8 [CICD]fix image name in bigdata toolkit (#7978) 2023-04-03 15:57:27 +08:00
Heyang Sun
e91cb31575 set no (#7974) 2023-04-03 09:02:43 +08:00
Shaojie Cui
13717ee5c8 [CICD]add noattest bigdata toolkit image (#7968) 2023-03-31 14:02:08 +08:00
Shaojie Cui
522e5ae35b [CICD]add no_proxy arg when building bigdata (#7868)
* [CICD]add no_proxy arg when building bigdata

* fix
2023-03-14 16:34:52 +08:00
Shaojie Cui
57125dfcd4 [PPML]tag 8g epc bigdata as default image (#7789)
* build 4g and tag 8g as latest

* reduce script memory

* doc
2023-03-08 10:49:40 +08:00
Shaojie Cui
f0fa26a8a8 [PPML]Reduce the epc usage of bigdata (#7606)
* build bigdata with smaller epc

* reduce epc in script

* fix action

* install pyarrow

* delete unused part

* reduce epc

* fix

* malloc_arena_max

* set 16g to default and delete 4g

* unused example

* spark simplequery

* pyspark simplequery

* fix

* fix

* fix

* code style fix
2023-03-01 14:59:27 +08:00
Guancheng Fu
3a57b97c2b Add dl-serving manually_build workflow (#7719) 2023-03-01 11:05:57 +08:00
Shaojie Cui
b12f2e1902 [CICD]delete bigdata image with debug log level (#7657) 2023-02-23 09:53:06 +08:00
Shaojie Cui
e5fb0a3315 [CICD]build bigdata toolkit with no cache (#7652) 2023-02-22 16:32:28 +08:00
Shaojie Cui
ec2b9c13ee [CICD]add bigdata toolkit with debug log level (#7564) 2023-02-15 20:16:17 +08:00
Heyang Sun
192a4d4c68 fix variable typo (#7543) 2023-02-14 17:18:20 +08:00
Shaojie Cui
670c54c6d7 [PPML]fix:add image with tag 'latest' in private dockerfile (#7441) 2023-02-06 23:01:16 +08:00
Heyang Sun
338423bfc9 Add bigdl-kms image build action 2023-01-11 09:42:30 +08:00
Shaojie Cui
3e040d7b28 [PPML]refactor bigdata build (#7088) 2022-12-26 13:49:59 +08:00
Heyang Sun
7c3d2b835d add 16G reference gramine images 2022-12-26 09:30:42 +08:00
Wang Jian
b82975a8d1 [PPML] Update customer image readme (#6859)
* push no build production image

* init build.sh

* update

* update path

* update rm order

* update path

* update readme

* update
2022-12-02 16:45:30 +08:00
Xiangyu Tian
cd8d21dded Fix docker rmi in manually_build (#6716) 2022-11-23 13:44:48 +08:00
Shaojie Cui
aff0f55b7f fix: add var base_image in bigdata build (#6616) 2022-11-15 16:31:33 +08:00
Shaojie Cui
f1d4fb4d37 cicd:add option bigdl-ppml-trusted-bigdata-gramine (#6615) 2022-11-15 16:19:43 +08:00
Heyang Sun
dabea499ff fix kmsutils action bug 2022-11-14 16:05:33 +08:00
Heyang Sun
2d7ea756af fix occlum-customer 2022-11-14 16:01:51 +08:00
Wang Jian
348c6aa092 [PPML] Add occlum manually customer image action build (#6588)
* fix no word '\'

* add manually customer image build action

* update build path
2022-11-14 13:47:17 +08:00
Jin Hanyu
93351ffd0a Add python toolkit image to manually_build.yml. (#6581) 2022-11-14 10:42:45 +08:00
Wang Jian
a9c3c18108 [PPML] Split occlum build and run for production (#6522)
* add mount.sh to mount k8s runtime config

* add mount conf and mkdir mount path

* run.sh init

* entrypoint bash mount

* init bash add_conf.sh

* init dockerfile

* dockerfile add ../file

* add mount.sh and add_conf.sh to dockerfile

* move register and attest

* docker and k8s mount and attest

* occlum-build.sh init

* update default name

* default attest and enable sgx debug

* add move files

* change ../ tp .

* Revert "no"

This reverts commit bd09e193e47c2ed72b10299126df5bcc7784f3da.

* update build image dir

* update build spell

* -it to -i

* update production manually build

* update manually push image name

* update register error message
2022-11-11 14:12:42 +08:00
Heyang Sun
7302451234 add debug image for azure 2022-11-08 16:39:29 +08:00
Heyang Sun
8b41130ca0 add 16g gramine image 2022-11-08 15:14:13 +08:00
Heyang Sun
ff69b05616 Update manually_build.yml 2022-11-08 15:06:05 +08:00
Heyang Sun
f9f2b9614b fix bug 2022-11-04 12:00:06 +08:00
Heyang Sun
5c5066b2c0 add debug log level image in noattest 2022-11-04 11:44:56 +08:00
Heyang Sun
1497bde87a add debug log level gramine images 2022-11-04 10:17:26 +08:00
Heyang Sun
d911f89b8d add debug log level image 2022-11-04 10:16:50 +08:00
Heyang Sun
23d79f7b7b Update manually_build.yml 2022-11-02 12:13:35 +08:00
Heyang Sun
ed3cc9ed64 set no-attest flag in image build console 2022-10-31 14:34:39 +08:00
Shaojie Cui
13f2557785 [PPML]CI/CD: build 64g Bigdata Toolkit (#6348) 2022-10-28 14:51:24 +08:00
Guancheng Fu
97b5c6a813 [PPML] fix mnist.py (#6320)
* fix mnist.py

* Add default SGX_LOG_LEVEL to error in workflow
2022-10-27 11:14:14 +08:00
Guancheng Fu
8a15a79b1a [PPML] Modify image to support pert model (#6293) 2022-10-26 11:48:48 +08:00
Shaojie Cui
6e32cfc215 [PPML]add gramine bigdata toolkit (#6135)
* java scala spark hadoop hive

* flink

* tmp copy

* bigdata toolkit

* change image tag

* change build script

* typo

* add log4j2.xml

* change https to http

* fix

* exchange location of spark and flink

* add maven proxy

* change proxy name

* update settings

* add scala version arg

* add submit.sh

* delete test script

* change home and add dependencies

* add custom images

* fix

* add sgx related files

* path change

* arg move

* move unused env and directory

* remove scala

* remove settings

* upgrade spark

* delete settings

* delete work directory

* modify path

* remove java

* remove java in first stage

* readd jdk in first stage

* fix

* build base image script

* add readme

* remove flink

* fix

* remove unused env

* CI/CD

* trigger test

* Trigger test

* fix proxy

* enable non-privilege

* remove directories in level 1

* change resuorces

* remove test code

* change resources limits
2022-10-25 09:44:34 +08:00
Heyang Sun
70468a96c6 Fix ra config 2022-10-22 20:32:06 +08:00
Heyang Sun
f7e07ecc69 add no-attestation gramine images 2022-10-21 16:20:30 +08:00
shaojie
57493194e7 fixed docker push image (#6235) 2022-10-21 10:44:31 +08:00