Adds mediapipe_text package (#12)

* adds mediapipe_text package

* Update .vscode/settings.json

added newline

* resync headers

* regenerated core bindings

* Apply suggestions from code review

* Adds example to `mediapipe-task-text` (#15)

* initial commit of example

* build file changes from `flutter pub get`

* update main.dart

* removes commented code

* updates example for isolates design

* Use `native-assets` to vendor MediaPipe SDK (#9)

* adding bare structure for native assets

* add MVP / first draft of build.dart

* build.dart updates

* update build.dart

TODO: stream file

* model memory troubleshooting

* vendoring script tweak

* remove development logging

* removes pointless build method

* Add utility to collect headers from google/mediapipe (#10)

* adds cmd to pull header files from google/mediapipe

* polish and missing parts from git surgery

* More comments and touch ups

* Apply suggestions from code review

* moves build command into `tool/` directory

and renames folder `build_cmd` -> `builder`

* complete build_cmd -> builder rename

* Update readme

* Added licenses

* Adds DownloadModelCommand

---------

Co-authored-by: Kate Lovett <katelovett@google.com>

* adds mediapipe_text package

* Update .vscode/settings.json

added newline

* resync headers

* regenerated core bindings

* native assets troubleshooting

this commit is broken

* Removes redundant count field

* update build.dart for correct bindings path

* download text classification model for CI

* better memory freeing in executor

* added SafeArea to example

* added CI to PRs into text package

* ci tooling change

* remove accidentally commited model

* more CI shenanigans

* lowers minimum Dart version for builder

* added smoke test for text example

* Added CI/CD for examples

* More CI tweaks

* entering "please work" territory

* d'oh

* trying more random stuff

* one more time

* it'd be funny if this helped

* more print statements

* enable reaching new print statements

* more logging

* see what's in build dir

* another test

* adding flutter config list

* turn off fail-fast for beta and master

* moar logs

* way moar prints

* moare things

* commit rest of rename

* moar whatevers

* adds manifest files generated by new sdks_finder command

* adds sdks_finder command to builder utility

* propagates changes to existing commands

* updates in response to code review

* updates to build.dart and tests

* add Android runtime

* sdks_finder logging improvement for when build folders change names

* refreshed symbols from google/mediapipe

* cleanup

* loosens closeness thresholds in integration tests

* separate build commands for macos architectures

* restores fail-fast setting to CI

* removed stale logging statements from CI

* removes accidentally committed lines

* add formatting of sdk_downloads.dart for CI

* fixes broken example test

* code touch ups from @Piinks code review

---------

Co-authored-by: Kate Lovett <katelovett@google.com>

* added base Dart class ClassificationResult to consolidate results logic in `mediapipe_core`

in doing so, removed meaningless TextClassifierResult.timestamp field

* Update Makefile

* Update Makefile

* Update packages/mediapipe-task-text/build.dart

Co-authored-by: Kate Lovett <katelovett@google.com>

* Update packages/mediapipe-task-text/lib/src/tasks/text_classification/text_classification_executor.dart

Co-authored-by: Kate Lovett <katelovett@google.com>

* code review responses

* formatting

* removes stale comment

* adds Dart to Native converters, with tests

* changes from code review

* updates mediapipe-core to prepare for IO/web split

* Improves memory management in core tests

* updated ffigen / bindings

* refactors text package for better memory management and eventual web/io split

* removed stale test

* cleanup on aisle COMMENTS

* Comments and documentation improvements

* Moved log statement

* Removed native memory management helpers in favor of `free` extensions on pointers

* Renamed abstract classes to have Base prefix

* sorted out class constructors

* moved `fake` constructor to default unnamed constructor
* leaning on the fact that the native constructors will be hidden by conditional exports, reducing confusion

* Improved docstrings explaining memory ownership

* Convert lists to lazy iterable / generators

* added missing licenses

* Update packages/mediapipe-task-text/example/test/widgets_test.dart

Co-authored-by: Kate Lovett <katelovett@google.com>

* Update packages/mediapipe-task-text/example/lib/main.dart

Co-authored-by: Kate Lovett <katelovett@google.com>

* completed return style change

* CI troubleshooting

* moved around debugging code

* logging tweak

* cat native-assets.yaml

* moar logs

* removed bad echo

* fixed native-assets typo

should be underscore!

* Removes CI debugging statements

---------

Co-authored-by: Kate Lovett <katelovett@google.com>

.github/workflows/main.yaml

@@ -7,7 +7,7 @@ on:
   push:
     branches: [main]
   pull_request:
-    branches: [main, ffi-wrapper, ffi-wrapper-core-pkg]
+    branches: [main, ffi-wrapper, ffi-wrapper-text-pkg]
   workflow_dispatch:
   schedule:
     - cron: "0 0 * * *" # Every day at midnight
@@ -25,7 +25,9 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        flutter_version: [stable, beta, master]
+        flutter_version: [master]
+        # TODO(craiglabenz): reverse once CI works
+        # flutter_version: [stable, beta, master]
         # TODO(craiglabenz): Add `ubuntu-latest` and `windows-latest` when those artifacts exist
         os: [macos-latest]
     steps:

.vscode/settings.json

@@ -0,0 +1,8 @@
+{
+    "files.associations": {
+        "base_options.h": "c",
+        "__config": "c",
+        "text_classifier.h": "c",
+        "category.h": "c"
+    }
+}

Makefile

@@ -1,20 +1,34 @@
 # Runs the utility to pull in all header files from `google/mediapipe`
 headers:
-	cd packages/build_cmd && dart bin/main.dart headers
+	cd tool/builder && dart bin/main.dart headers
 
 # Runs `ffigen` for all packages
 generate: generate_core generate_text
 
-# Runs `ffigen` for all packages, compiles the faked C artifacts, and runs all tests
-test: generate_text test_text generate_core test_core
+# Runs `ffigen` for all packages and runs all tests
+test: generate_core test_core generate_text test_text
 
-# Runs `ffigen` for all packages and all tests for all packages
+# Runs all tests for all packages
 test_only: test_core test_text
 
+# Rebuilds the MediaPipe task for macOS
+# Assumes google/mediapipe and google/flutter-mediapipe are siblings on the file system
+compile_text_classifier_macos_arm:
+	cd ../mediapipe && bazel build --linkopt -s --config darwin_arm64 --strip always --define MEDIAPIPE_DISABLE_GPU=1 mediapipe/tasks/c/text/text_classifier:libtext_classifier.dylib
+	cd ../mediapipe && sudo cp bazel-bin/mediapipe/tasks/c/text/text_classifier/libtext_classifier.dylib ../flutter-mediapipe/packages/mediapipe-task-text/example/assets/libtext_classifier_arm64.dylib
+
+compile_text_classifier_macos_x86:
+	cd ../mediapipe && bazel build --linkopt -s --config darwin_x86_64 --strip always --define MEDIAPIPE_DISABLE_GPU=1 mediapipe/tasks/c/text/text_classifier:libtext_classifier.dylib
+	cd ../mediapipe && sudo cp bazel-bin/mediapipe/tasks/c/text/text_classifier/libtext_classifier.dylib ../flutter-mediapipe/packages/mediapipe-task-text/example/assets/libtext_classifier_x64.dylib
+
 # Runs `sdks_finder` to update manifest files
 sdks:
 	dart tool/builder/bin/main.dart sdks
 
+analyze:
+	cd packages/mediapipe-core && dart format -o write .
+	cd packages/mediapipe-task-text && dart format -o write .
+
 # Core ---
 
 # Runs `ffigen` for `mediapipe_core`
@@ -29,19 +43,12 @@ test_core:
 
 # Runs `ffigen` for `mediapipe_text`
 generate_text:
-	cd packages/mediapipe-task-text && dart run ffigen --config=ffigen.yaml
-
-# Compiles the faked C artifacts for testing
-compile_fake_text:
-	# Builds standalone executable
-	cd packages/mediapipe-task-text/test/c && gcc fake_text_classifier.c -o fake_text_classifier
-	# Builds what Flutter needs
-	cd packages/mediapipe-task-text/test/c && gcc -static -c -fPIC *.c -o fake_text_classifier.o
-	cd packages/mediapipe-task-text/test/c && gcc -shared -o fake_text_classifier.dylib fake_text_classifier.o 
-
-# Runs `ffigen` for `mediapipe_text` and all text tests
-test_text: compile_fake_text test_text_only
+	cd packages/mediapipe-task-text && dart --enable-experiment=native-assets run ffigen --config=ffigen.yaml
 
 # Runs all text tests
-test_text_only:
+test_text:
 	cd packages/mediapipe-task-text && flutter test
+	cd packages/mediapipe-task-text/example && flutter test
+
+example_text:
+	cd packages/mediapipe-task-text/example && flutter run -d macos

packages/mediapipe-core/README.md

@@ -24,4 +24,4 @@ Issues that are specific to Flutter can be filed in the [Flutter issue tracker](
 
 To contribute a change to this plugin,
 please review our [contribution guide](https://github.com/google/flutter-mediapipe/blob/master/CONTRIBUTING.md)
-and open a [pull request](https://github.com/google/flutter-mediapipe/pulls).
+and open a [pull request](https://github.com/google/flutter-mediapipe/pulls).

packages/mediapipe-core/analysis_options.yaml

@@ -1,5 +1,9 @@
 include: ../analysis_options.yaml
 
+linter:
+  rules:
+    - public_member_api_docs # see https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo#documentation-dartdocs-javadocs-etc
+
 analyzer:
   exclude:
     - "**/mediapipe_common_bindings.dart"

packages/mediapipe-core/ffigen.yaml

@@ -1,13 +1,13 @@
 name: "MediaPipeCommonBindings"
 description: "Bindings for shared MediaPipe structs common across many tasks"
 output:
-  bindings: "lib/src/third_party/mediapipe/generated/mediapipe_common_bindings.dart"
+  bindings: "lib/src/io/third_party/mediapipe/generated/mediapipe_common_bindings.dart"
   symbol-file:
     output: "package:mediapipe_core/generated/core_symbols.yaml"
-    import-path: "package:mediapipe_core/src/third_party/mediapipe/generated/mediapipe_common_bindings.dart"
+    import-path: "package:mediapipe_core/src/io/third_party/mediapipe/generated/mediapipe_common_bindings.dart"
 headers:
   entry-points:
-    - "third_party/mediapipe/tasks/c/**"
+    - "third_party/mediapipe/tasks/c/**.h"
 preamble: |
   /* Copyright 2023 The MediaPipe Authors.
 

packages/mediapipe-core/lib/generated/core_symbols.yaml

@@ -1,11 +1,13 @@
 format_version: 1.0.0
 files:
-  package:mediapipe_core/src/third_party/mediapipe/generated/mediapipe_common_bindings.dart:
+  package:mediapipe_core/src/io/third_party/mediapipe/generated/mediapipe_common_bindings.dart:
     used-config:
       ffi-native: false
     symbols:
       c:@S@BaseOptions:
         name: BaseOptions
+      c:@S@Categories:
+        name: Categories
       c:@S@Category:
         name: Category
       c:@S@ClassificationResult:

packages/mediapipe-core/lib/interface.dart

@@ -0,0 +1,5 @@
+// Copyright 2014 The Flutter Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+export 'src/interface/interface.dart';

packages/mediapipe-core/lib/io.dart

@@ -0,0 +1,5 @@
+// Copyright 2014 The Flutter Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+export 'src/io/mediapipe_core.dart';

packages/mediapipe-core/lib/mediapipe_core.dart

@@ -4,8 +4,9 @@
 
 /// Package containing core dependencies for MediaPipe's text, vision, and
 /// audio-based tasks.
-library;
+library mediapipe_core;
 
-export 'src/containers.dart' show Category, Classifications;
-export 'src/ffi_utils.dart';
-export 'src/task_options.dart' show BaseOptions, ClassifierOptions;
+export 'src/extensions.dart';
+export 'universal_mediapipe_core.dart'
+    if (dart.library.html) 'src/web/mediapipe_core.dart'
+    if (dart.library.io) 'src/io/mediapipe_core.dart';

packages/mediapipe-core/lib/src/extensions.dart

@@ -0,0 +1,14 @@
+// Copyright 2014 The Flutter Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+/// Shortens long strings for logging, printing, etc.
+extension DebuggableString on String {
+  /// Shortens the string to its first X characters, replacing the rest with an
+  /// ellipses and the total length. If the string is equal to or shorter than
+  /// the given length, then [shorten] is a no-op.
+  String shorten([int x = 10]) {
+    if (length <= x) return this;
+    return '${substring(0, x)}...[$length]';
+  }
+}