From 318cf481d8d240b533f7fc2ccb4901b01913146e Mon Sep 17 00:00:00 2001 From: Ryan Bogan Date: Fri, 20 Oct 2023 10:46:24 -0700 Subject: [PATCH] Update developer guide to include M1 Setup (#1222) * Update Developer Guide to include M1 setup Signed-off-by: Ryan Bogan Co-authored-by: Heemin Kim --- CHANGELOG.md | 1 + DEVELOPER_GUIDE.md | 51 +++++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 51 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 4c7a9cce1..4e4167ae7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -20,4 +20,5 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), ### Infrastructure ### Documentation ### Maintenance +* Update developer guide to include M1 Setup [#1222](https://github.com/opensearch-project/k-NN/pull/1222) ### Refactoring diff --git a/DEVELOPER_GUIDE.md b/DEVELOPER_GUIDE.md index 4fe31487a..582041e1d 100644 --- a/DEVELOPER_GUIDE.md +++ b/DEVELOPER_GUIDE.md @@ -63,6 +63,11 @@ One easy way to install on mac or linux is to use pip: pip install cmake==3.23.3 ``` +On Mac M1 machines, install cmake using: +```bash +brew install cmake +``` + #### Faiss Dependencies To build the *faiss* JNI library, you need to have openmp, lapack and blas installed. For more information on *faiss* @@ -78,6 +83,50 @@ Additionally, the `gcc` toolchain needs to be installed on Mac. To install, run: brew install gcc ``` +#### Extra setup for Mac M1 Machines + +The following commands enable running/building k-NN on M1 machines: + +```bash +// Go to k-NN folder +cd k-NN + +// Build to generate the necessary files to be modified below (will fail) +./gradlew build + +//Go to jni folder +cd jni + +// File changes required +sed -i -e 's/\/usr\/local\/opt\/libomp\//\/opt\/homebrew\/opt\/llvm\//g' CMakeLists.txt +sed -i -e 's/-march=native/-mcpu=apple-m1/g' external/nmslib/similarity_search/CMakeLists.txt +sed -i -e 's/pragma message WARN/pragma message /g' external/nmslib/similarity_search/src/distcomp_scalar.cc +sed -i -e 's/-mcpu=apple-a14/-mcpu=apple-m1/g' external/nmslib/python_bindings/setup.py +sed -i -e 's/__aarch64__/__undefine_aarch64__/g' external/faiss/faiss/utils/distances_simd.cpp + +// Install llvm +brew install llvm +echo 'export PATH="/opt/homebrew/opt/llvm/bin:$PATH"' >> ~/.zshrc +source ~/.zshrc + +// Set compiler path for CMAKE +export CC=/opt/homebrew/opt/llvm/bin/clang +export CXX=/opt/homebrew/opt/llvm/bin/clang++ + +// Build +cmake . --fresh +make +``` + +Next, obtain a minimum distribution tarball of the k-NN version you want to build: + +1. Fork the [OpenSearch Repo](https://github.com/opensearch-project/OpenSearch) into your github account. +2. Clone the repository locally +3. Run the following commands: +```cd OpenSearch && ./gradlew -p distribution/archives/darwin-tar assemble``` +4. You should see a opensearch-min--SNAPSHOT-darwin-x64.tar.gz file present in distribution/archives/darwin-tar/build/distributions/ +5. Build k-NN by passing the OpenSearch distribution path in `./gradlew -PcustomDistributionUrl=""` + #### Environment Currently, the plugin only supports Linux on x64 and arm platforms. @@ -114,7 +163,7 @@ Please follow these formatting guidelines: * Wildcard imports (`import foo.bar.baz.*`) are forbidden and will cause the build to fail. * If *absolutely* necessary, you can disable formatting for regions of code with the `// tag::NAME` and `// end::NAME` directives, but note that these are intended for use in documentation, so please make it clear what you have done, and only do this where the benefit clearly outweighs the decrease in consistency. * Note that JavaDoc and block comments i.e. `/* ... */` are not formatted, but line comments i.e `// ...` are. -* There is an implicit rule that negative boolean expressions should use the form `foo == false` instead of `!foo` for better readability of the code. While this isn't strictly enforced, if might get called out in PR reviews as something to change. +* There is an implicit rule that negative boolean expressions should use the form `foo == false` instead of `!foo` for better readability of the code. While this isn't strictly enforced, it might get called out in PR reviews as something to change. ## Build