Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update developer guide to include M1 Setup #1222

Merged
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,4 +19,5 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
### Infrastructure
### Documentation
### Maintenance
- Update developer guide to include M1 Setup [#1222](https://github.com/opensearch-project/k-NN/pull/1222)
ryanbogan marked this conversation as resolved.
Show resolved Hide resolved
### Refactoring
45 changes: 44 additions & 1 deletion DEVELOPER_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,11 @@ One easy way to install on mac or linux is to use pip:
pip install cmake==3.23.3
```

On Mac M1 machines, install cmake using:
```bash
brew install cmake
ryanbogan marked this conversation as resolved.
Show resolved Hide resolved
```

#### Faiss Dependencies

To build the *faiss* JNI library, you need to have openmp, lapack and blas installed. For more information on *faiss*
Expand All @@ -78,6 +83,44 @@ Additionally, the `gcc` toolchain needs to be installed on Mac. To install, run:
brew install gcc
```

#### Extra setup for Mac M1 Machines

The following commands enable running/building k-NN on M1 machines:

```bash
// Go to jni folder
cd k-NN/jni

// File changes required
sed -i -e 's/\/usr\/local\/opt\/libomp\//\/opt\/homebrew\/opt\/llvm\//g' CMakeLists.txt
sed -i -e 's/-march=native/-mcpu=apple-m1/g' external/nmslib/similarity_search/CMakeLists.txt
sed -i -e 's/pragma message WARN/pragma message /g' external/nmslib/similarity_search/src/distcomp_scalar.cc
sed -i -e 's/-mcpu=apple-a14/-mcpu=apple-m1/g' external/nmslib/python_bindings/setup.py
sed -i -e 's/__aarch64__/__undefine_aarch64__/g' external/faiss/faiss/utils/distances_simd.cpp
ryanbogan marked this conversation as resolved.
Show resolved Hide resolved

// Install llvm
brew install llvm
echo 'export PATH="/opt/homebrew/opt/llvm/bin:$PATH"' >> ~/.zshrc
source ~/.zshrc

// Set compiler path for CMAKE
export CC=/opt/homebrew/opt/llvm/bin/clang
export CXX=/opt/homebrew/opt/llvm/bin/clang++

// Build
cmake . --fresh
make
```

Next, obtain a minimum distribution tarball of the k-NN version you want to build:

1. Fork the [OpenSearch Repo](https://github.com/opensearch-project/OpenSearch) into your github account.
2. Clone the repository locally
3. Run the following commands:
```cd OpenSearch && ./gradlew -p distribution/archives/darwin-tar assemble```
4. You should see a opensearch-min-<version>-SNAPSHOT-darwin-x64.tar.gz file present in distribution/archives/darwin-tar/build/distributions/
5. Build k-NN by passing the OpenSearch distribution path in `./gradlew <integTest/run> -PcustomDistributionUrl="<Full path to .tar.gz file you noted above>"`
ryanbogan marked this conversation as resolved.
Show resolved Hide resolved

#### Environment

Currently, the plugin only supports Linux on x64 and arm platforms.
Expand Down Expand Up @@ -114,7 +157,7 @@ Please follow these formatting guidelines:
* Wildcard imports (`import foo.bar.baz.*`) are forbidden and will cause the build to fail.
* If *absolutely* necessary, you can disable formatting for regions of code with the `// tag::NAME` and `// end::NAME` directives, but note that these are intended for use in documentation, so please make it clear what you have done, and only do this where the benefit clearly outweighs the decrease in consistency.
* Note that JavaDoc and block comments i.e. `/* ... */` are not formatted, but line comments i.e `// ...` are.
* There is an implicit rule that negative boolean expressions should use the form `foo == false` instead of `!foo` for better readability of the code. While this isn't strictly enforced, if might get called out in PR reviews as something to change.
* There is an implicit rule that negative boolean expressions should use the form `foo == false` instead of `!foo` for better readability of the code. While this isn't strictly enforced, it might get called out in PR reviews as something to change.

## Build

Expand Down
Loading