How to setup OpenCV (for iOS apps)

OpenCV is an open-source image processing and machine learning library. Its highly functional library allows you to develop applications more efficiently than implementing all image processing and machine learning processes from scratch.

Not only is it highly functional, but it is also speedy. Internally, OpenCL, SIMD, IPP (Intel Performance Primitive), and threaded parallelization are used to achieve speeds that cannot be obtained with ordinary, straightforward implementation.

This article describes how to set up an OpenCV development environment to develop iOS applications using OpenCV.

For more information on how to set up a development environment for macOS applications, see the following article.

目次

Xcode project settings

To use OpenCV in your app, embed the downloaded framework. Set up your project as follows.

Download SDK

The iOS version of the SDK is available as a pre-built framework on GitHub’s Release page. Download this.

Copy the framework

The opencv2.framework you downloaded includes header files. Arrange these into the following folder structure.

OpenCVTest_iOS
├── OpenCVTest_iOS
│   ├── Assets.xcassets
│   ├── ContentView.swift
│   ├── OpenCVTest_iOSApp.swift
│   └── Preview Content
├── OpenCVTest_iOS.xcodeproj
│   ├── project.pbxproj
│   ├── project.xcworkspace
│   └── xcuserdata
└── common
    └── opencv2.framework

Add the framework

Add the opencv2.framework to your project. Operate as follows.

STEP
Drag and drop the “common” folder into the project.
Drag and drop the "common" folder.
Drag and drop the “common” folder
STEP
Select “Create groups”, check the application targets, and click the “Finish” button.
Select the options and click the "Finish" button
Select the options and click the “Finish” button

Configure framework embedding

Operate as follows.

STEP
Open the application’s target settings and open the “General” tab.
STEP
Select “Do Not Embed” for the “Embed” setting of “opencv2.framework” in “Frameworks, Libraries and Embedded Content”.
Select the "Do Not Embed"
Select the “Do Not Embed”

Error regarding the binary type

The following error may appear. In our case, this occurred with Xcode 13.4.1 + OpenCV 4.6.

Buidling for iOS Simulator, but the linked and embedded framework 'opencv2.framework' was built for iOS + iOS Simulator
Error regarding binary type is displayed
Error regarding binary type is displayed

The error message indicates that opencv2.framework encompasses two binaries – one for the iOS device and the other for the iOS simulator – and the app targets the iOS simulator. Initially, I did not perceive this as an issue.

This may seem like a bug in Xcode, but you can work around it by doing the following.

STEP
Change “Validate Workspace” to “YES” under “Build Options” in the build settings.
STEP
Change the value that was changed to “YES” in step (1) to “NO”.
Change "Validate Workspace" to "YES" and then back to "NO
Change “Validate Workspace” to “YES” and then back to “NO

This error appears to be an error that occurs when “Validate Workspace” is YES. However, it was initially displayed as “NO”, meaning it would be “NO” when nothing was set. (not bolded)

Changing it once and then reverting it will explicitly set it to specify “NO”.

This is an expectation, but Xcode assumes that the default setting is “NO”, but the linker’s default value may be “YES”.

Test

Let’s try the following simple code to see if OpenCV works: OpenCV is C++, so we write it in Objective-C++ and call it from Swift. The image to be used is embedded in a resource, so it is loaded with UIImage.

// OpenCVTest.h

#import <UIKit/UIKit.h>

@interface OpenCVTest : NSObject
+ (nullable UIImage *)filteredImage;
@end
// OpenCVTest.mm

#import "opencv2/opencv.hpp"
#import "opencv2/imgproc.hpp"
#import "opencv2/imgcodecs.hpp"
#import "opencv2/imgcodecs/ios.h"
#import "OpenCVTest.h"

@implementation OpenCVTest

+ (nullable UIImage *)filteredImage
{
    UIImage *srcImage = [UIImage imageNamed:@"P4071145"];
    cv::Mat srcImageMat;
    cv::Mat dstImageMat;
    
    // Convert from UIImage to cv::Mat
    UIImageToMat(srcImage, srcImageMat);
    
    // Convert the color space from RGB to Gray
    cv::cvtColor(srcImageMat, dstImageMat, cv::COLOR_RGB2GRAY);
    
    // Convert from cv::Mat to UIImage
    UIImage *dstImage = MatToUIImage(dstImageMat);
    
    return dstImage;
}

@end

Include OpenCVTest.h in a bridging header to use Objective-C++ classes from the Swift code. The bridging header is created with the name ProjectName-Bridging-Header.h. In this project, it is OpenCVTest_iOS-Bridging-Header.h. Write the following code in the bridging header.

#import "OpenCVTest.h"

The bridging header is specified in the Objective-C Briding Header under Swift Compiler - General in the target build settings.

Specify the Objective-C Bridging Header
Specify the Objective-C Bridging Header

Next, invoke it from the Swift code. We executed this procedure using SwiftUI. Here is the corresponding code:

//  ContentView.swift

import SwiftUI

struct ContentView: View {
    var filteredImage: UIImage? = {
        OpenCVTest.filteredImage()
    }()
    
    var body: some View {
        ZStack {
            if self.filteredImage != nil {
                Image(uiImage: filteredImage!)
                    .resizable()
                    .aspectRatio(filteredImage!.size, contentMode: .fit)
            } else {
                Text("No Image")
            }
        }
    }
}

struct ContentView_Previews: PreviewProvider {
    static var previews: some View {
        ContentView()
    }
}

Test on the device

Let’s run it on a real device. The following screen capture is when run on an iPhone 13 Pro. It is nicely grayscaled.

Running results on iPhone 13 Pro
Running results on iPhone 13 Pro

Running results on the simulator and preview

When we tried to run the simulator with a pre-built binary of OpenCV 4.6.0, we got the following error.

ld: in /Volumes/Data/src/RK/TechGakuWebSite/SampleCodes/OpenCVTest_iOS/common/opencv2.framework/opencv2(ios_conversions.o), building for iOS Simulator, but linking in object file built for iOS, for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

This error emerges when you are building for an iOS simulator but linking to an iOS device. For Apple Silicon machines such as the M1, the binary is arm64. Therefore, the build for the simulator also uses the arm64 version, i.e., for iOS devices, so it is an error.

It works by changing the binary for the iOS simulator to x86_64. The simulator on the Apple Silicon machine also supports the x86_64 binary by Rosetta2.

See the following article.

著書紹介

Authored Books

よかったらシェアしてね!
  • URLをコピーしました!
  • URLをコピーしました!

この記事を書いた人

Akira Hayashi (林 晃)のアバター Akira Hayashi (林 晃) Representative(代表), Software Engineer(ソフトウェアエンジニア)

アールケー開発代表。Appleプラットフォーム向けの開発を専門としているソフトウェアエンジニア。ソフトウェアの受託開発、技術書執筆、技術指導・セミナー講師。note, Medium, LinkedIn
-
Representative of RK Kaihatsu. Software Engineer Specializing in Development for the Apple Platform. Specializing in contract software development, technical writing, and serving as a tech workshop lecturer. note, Medium, LinkedIn

目次