Commit 9ecf5b64 authored by Drew's avatar Drew

Add proper platform support

Related to #36

Presently, we tend to enable platform-specific config with an overlay.
There are a variety of problems identified with this approach:

1.  There is no convention for which overlay to use for platform-
specific config.  This complicates the ecosystem.

2.  In general, a program is always compiled "for some platform" but in
practice a user may forget the necessary overlay.  `require-overlays`
can catch this misconfig, but A) it has to be opted into in the atpkg,
and B) there is no good way to default to the running platform, which is
the sane behavior.

3.  Currently, tools tend to conditionally-compile platform-specific
code.  The effect of this is to complicate bootstrapping, as a Linux
binary *cannot* perform some OSX behavior (and vice versa) because the
necessary code is not present in the executable.

4.  This is not scaleable to cross-compilation (iOS for example is
Coming Soon but can't be supported in this architecture)

To work around these problems, we introduce a `Platform` enum, to
replace `PlatformPaths`.  `Platform` is a runtime technology, allowing
for a Linux binary to reason about what the behavior would be on OSX
etc.

Internally, we have three "platforms":

* `hostPlatform`, the platform on which `atbuild` is currently executing
* `targetPlatform`, the platform for which we are compiling. By default
   this is the `hostPlatform`
* `buildPlatform`, the platform where `swift-build-tool` will run.
   This is usually the `hostPlatform`, but if we are bootstrapping it
   is the `targetPlatform` instead.

The user can override the `targetPlatform` by the use of `--platform
foo`.  `linux` and `osx` are supported.  `mac` is supported as an alias
of `osx`.

The primary effect of a platform is to scope tool-specific behavior
(e.g., target=OSX uses the OSX SDK, host=Linux uses a linux path for the
toolchain, etc).

In addition to the tool-specific behavior, we enable overlays for the
target platform:

* `atbuild.platform.linux` on Linux
* `atbuild.platform.osx` and `atbuild.platform.mac` on OSX

This allows packages to reliably perform per-platform configuration in
the overlays.  Critically, some platform overlay is reliably active, so
users in most cases will not have to `--use-overlay` to get proper
platform behavior.

DEPRECATION WARNING: We believe the `swiftc-path` key is no longer
required, as the functionality used can be achieved either by
`--toolchain` or `--platform`.  Therefore, I'm adding a warning to users
that we intend to remove it and to try these features instead.

We need to put out a release with these features (and the warning)
before I'm happy to remove it.  In particular, we use it inside
atbuild/atpkg, and removing it immediately would break bootstrapping, so
let's give it a little time before we tear it out.  We should remove it
from the documentation though.
parent 5090df20
Pipeline #1530 passed with stage
......@@ -75,11 +75,12 @@ That's all you need to get started! `atbuild` supports many more usecases than
`atbuild` supports several command-line options:
* `--use-overlay [overlay]`, which you can read more about in our [overlays](overlays.html) documentation.
* `--use-overlay [overlay]`, which you can read more about in our [overlays](http://anarchytools.org/docs/overlays.html) documentation.
* `-f [atpkg-file]` which builds a package file other than `build.atpkg`
* `--help`, which displays a usage message
* `--clean`, which forces a clean build
* `--toolchain` which specifies a nonstandard toolchain (swift installation). By default we try to guess, but you can override our guess here. The special string `xcode` uses "xcode swift" for building. (Swift 2.2 does not contain all the tools we use, so you need to have a 3.x snapshot installed as well. However, this is a "mostly" xcode-flavored buildchain.)
* `--platform` which specifies the target platform. By default, this is the current platform. Pass a different value (`osx` or `linux`) to cross-compile, only if your compiler supports it.
# Building
......
......@@ -27,24 +27,25 @@ enum Options: String {
case Help = "--help"
case Clean = "--clean"
case Toolchain = "--toolchain"
static var allOptions : [Options] { return [Overlay, CustomFile, Help, Clean, Toolchain] }
case Platform = "--platform"
static var allOptions : [Options] { return [
Overlay,
CustomFile,
Help,
Clean,
Toolchain,
Platform
]
}
}
let defaultPackageFile = "build.atpkg"
var focusOnTask : String? = nil
//build overlays
var overlays : [String] = []
for (i, x) in Process.arguments.enumerated() {
if x == Options.Overlay.rawValue {
let overlay = Process.arguments[i+1]
overlays.append(overlay)
}
}
var packageFile = defaultPackageFile
var toolchain = DefaultToolchainPath
var toolchain = Platform.buildPlatform.defaultToolchainPath
for (i, x) in Process.arguments.enumerated() {
if x == Options.CustomFile.rawValue {
packageFile = Process.arguments[i+1]
......@@ -55,7 +56,24 @@ for (i, x) in Process.arguments.enumerated() {
toolchain = "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain"
}
}
if x == Options.Platform.rawValue {
let platformString = Process.arguments[i+1]
Platform.targetPlatform = Platform(string: platformString)
}
}
//build overlays
var overlays : [String] = []
for (i, x) in Process.arguments.enumerated() {
if x == Options.Overlay.rawValue {
let overlay = Process.arguments[i+1]
overlays.append(overlay)
}
}
overlays.append(contentsOf: Platform.targetPlatform.overlays)
print("enabling overlays \(overlays)")
let package = try! Package(filepath: packageFile, overlay: overlays, focusOnTask: focusOnTask)
//usage message
......
......@@ -102,7 +102,19 @@ extension String {
func replacingOccurrences(of str: String, with: String) -> String {
return self.stringByReplacingOccurrencesOfString(str, withString: with)
}
func cString(usingEncoding encoding: NSStringEncoding) -> [CChar]? {
return self.cStringUsingEncoding(encoding)
}
init?(cString: UnsafePointer<CChar>, encoding: NSStringEncoding) {
precondition(encoding == NSUTF8StringEncoding)
self.init(validatingUTF8: cString)
}
func cString(using: NSStringEncoding) -> [CChar]? {
return self.cString(usingEncoding: using)
}
}
#endif
//These parts are possibly? not yet implemented on OSX
......
......@@ -68,7 +68,7 @@ class PackageFramework: Tool {
try! manager.createSymbolicLink(atPath: "\(frameworkPath)/Versions/Current", withDestinationPath: "A")
//copy payload
let payloadPath = task.importedPath + "bin/" + name + DynamicLibraryExtension
let payloadPath = task.importedPath + "bin/" + name + Platform.targetPlatform.dynamicLibraryExtension
print(payloadPath)
try! manager.copyItemAtPath_SWIFTBUG(srcPath: payloadPath, toPath: "\(AVersionPath)/\(name)")
try! manager.createSymbolicLink(atPath: "\(frameworkPath)/\(name)", withDestinationPath: "\(relativeAVersionPath)/\(name)")
......@@ -76,8 +76,8 @@ class PackageFramework: Tool {
//copy modules
let modulePath = "\(AVersionPath)/Modules/\(name).swiftmodule"
try! manager.createDirectory(atPath: modulePath, withIntermediateDirectories: true, attributes: nil)
try! manager.copyItemAtPath_SWIFTBUG(srcPath: "bin/\(name).swiftmodule", toPath: "\(modulePath)/\(Architecture).swiftmodule")
try! manager.copyItemAtPath_SWIFTBUG(srcPath: "bin/\(name).swiftdoc", toPath: "\(modulePath)/\(Architecture).swiftdoc")
try! manager.copyItemAtPath_SWIFTBUG(srcPath: "bin/\(name).swiftmodule", toPath: "\(modulePath)/\(Platform.targetPlatform.architecture).swiftmodule")
try! manager.copyItemAtPath_SWIFTBUG(srcPath: "bin/\(name).swiftdoc", toPath: "\(modulePath)/\(Platform.targetPlatform.architecture).swiftdoc")
try! manager.createSymbolicLink(atPath: "\(frameworkPath)/Modules", withDestinationPath: "\(relativeAVersionPath)/Modules")
//copy resources
......
......@@ -14,35 +14,119 @@
import Foundation
func findToolPath(toolName: String, toolchain: String) -> String {
//look in /usr/bin
let manager = NSFileManager.defaultManager()
let usrBin = "\(toolchain)/usr/bin/\(toolName)"
if manager.fileExists(atPath: usrBin) { return usrBin }
//look in /usr/local/bin
let usrLocalBin = "\(toolchain)/usr/local/bin/\(toolName)"
if manager.fileExists(atPath: usrLocalBin) { return usrLocalBin }
public enum Platform {
case OSX
case Linux
//swift-build-tool isn't available in 2.2.
//If we're looking for SBT, try in the default location
if toolName == "swift-build-tool" {
let sbtPath = "\(DefaultToolchainPath)/usr/bin/\(toolName)"
if manager.fileExists(atPath: sbtPath) { return sbtPath }
public init(string: String) {
switch(string) {
case "osx", "mac":
self = Platform.OSX
case "linux":
self = Platform.Linux
default:
fatalError("Unknown platform \(string)")
}
}
///The overlays that should be enabled when building for this platform
public var overlays: [String] {
switch(self) {
case .OSX:
return ["atbuild.platform.osx", "atbuild.platform.mac"]
case .Linux:
return ["atbuild.platform.linux"]
}
}
fatalError("Can't find a path for \(toolName)")
///The typical path to a toolchain binary of the platform
var defaultToolchainBinaryPath: String {
switch(self) {
case .OSX:
return "\(defaultToolchainPath)/usr/bin/"
case .Linux:
return "\(defaultToolchainPath)/usr/local/bin/"
}
}
public var defaultToolchainPath: String {
switch(self) {
case .OSX:
return "/Library/Developer/Toolchains/swift-latest.xctoolchain"
case .Linux:
return "/"
}
}
var sdkPath: String? {
switch(self) {
case .OSX:
return "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk"
case .Linux:
return nil
}
}
var architecture: String {
switch(self) {
case .OSX, .Linux:
return "x86_64"
}
}
var dynamicLibraryExtension: String {
switch(self) {
case .OSX:
return ".dylib"
case .Linux:
return ".so"
}
}
///The platform on which atbuild is currently running
public static var hostPlatform: Platform {
#if os(OSX)
return Platform.OSX
#elseif os(Linux)
return Platform.Linux
#endif
}
///The platform for which atbuild is currently building
///By default, we build for the hostPlatform
public static var targetPlatform: Platform = Platform.hostPlatform
///The platform on which the build will take place (e.g. swift-build-tool will run).
///Ordinarily we build on the hostPlatform, but in the case of bootstrapping,
///we may be only emitting a yaml, which the actual build occuring
/// on some other platform than either the host or the target.
public static var buildPlatform: Platform = Platform.hostPlatform
}
#if os(OSX)
let SDKPath = "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk"
public let DefaultToolchainPath = "/Library/Developer/Toolchains/swift-latest.xctoolchain"
let DynamicLibraryExtension = ".dylib"
let Architecture = "x86_64"
#elseif os(Linux)
let SwiftCPath = "/usr/local/bin/swiftc"
public let DefaultToolchainPath = "/"
let DynamicLibraryExtension = ".so"
let Architecture = "x86_64"
#endif
\ No newline at end of file
func findToolPath(toolName: String, toolchain: String) -> String {
if Platform.buildPlatform == Platform.hostPlatform {
//poke around on the filesystem
//look in /usr/bin
let manager = NSFileManager.defaultManager()
let usrBin = "\(toolchain)/usr/bin/\(toolName)"
if manager.fileExists(atPath: usrBin) { return usrBin }
//look in /usr/local/bin
let usrLocalBin = "\(toolchain)/usr/local/bin/\(toolName)"
if manager.fileExists(atPath: usrLocalBin) { return usrLocalBin }
//swift-build-tool isn't available in 2.2.
//If we're looking for SBT, try in the default location
if toolName == "swift-build-tool" {
let sbtPath = "\(Platform.hostPlatform.defaultToolchainPath)/usr/bin/\(toolName)"
if manager.fileExists(atPath: sbtPath) { return sbtPath }
}
}
else {
//file system isn't live; hope the path is in a typical place
return "\(Platform.buildPlatform.defaultToolchainBinaryPath)\(toolName)"
}
fatalError("Can't find a path for \(toolName)")
}
\ No newline at end of file
......@@ -23,7 +23,8 @@ class XCTestRun : Tool {
guard let testExecutable = task[Option.TestExecutable.rawValue]?.string else {
fatalError("No \(Option.TestExecutable.rawValue) for XCTestRun task \(task.qualifiedName)")
}
#if os(OSX)
switch (Platform.targetPlatform) {
case .OSX:
var workingDirectory = "/tmp/XXXXXXXXXXX"
var template = workingDirectory.cString(using: NSUTF8StringEncoding)!
workingDirectory = String(cString: mkdtemp(&template), encoding: NSUTF8StringEncoding)!
......@@ -66,12 +67,10 @@ class XCTestRun : Tool {
fatalError("Test execution failed.")
}
#elseif os(Linux)
case .Linux:
if system("\(testExecutable)") != 0 {
fatalError("Test execution failed.")
}
#else
fatalError("Not implemented")
#endif
}
}
}
\ No newline at end of file
......@@ -119,9 +119,9 @@ final class ATllbuild : Tool {
args.append(contentsOf: ["-j8", "-D","ATBUILD","-I",workdir+"products/"])
if linkSDK {
#if os(OSX) //we don't have SDKPath on linux
args.append(contentsOf: ["-sdk", SDKPath])
#endif
if let sdkPath = Platform.targetPlatform.sdkPath {
args.append(contentsOf: ["-sdk",sdkPath])
}
}
args.append(contentsOf: compileOptions)
......@@ -175,7 +175,7 @@ final class ATllbuild : Tool {
let builtProducts = linkWithProduct.map {workdir+"products/"+$0}
llbuild_inputs.append(contentsOf: builtProducts)
yaml += " inputs: \(llbuild_inputs)\n"
let libPath = productPath + modulename + DynamicLibraryExtension
let libPath = productPath + modulename + Platform.targetPlatform.dynamicLibraryExtension
yaml += " outputs: [\"<atllbuild>\", \"\(libPath)\"]\n"
var args = [swiftCPath, "-o", libPath, "-emit-library"]
args.append(contentsOf: objects)
......@@ -281,7 +281,7 @@ final class ATllbuild : Tool {
for product in arr {
guard var p = product.string else { fatalError("non-string product \(product)") }
if p.hasSuffix(".dynamic") {
p = p.replacingOccurrences(of: ".dynamic", with: DynamicLibraryExtension)
p = p.replacingOccurrences(of: ".dynamic", with: Platform.targetPlatform.dynamicLibraryExtension)
}
linkWithProduct.append(p)
}
......@@ -351,23 +351,31 @@ final class ATllbuild : Tool {
if task[Options.XCTestify.rawValue]?.bool == true {
precondition(outputType == .Executable, "You must use :\(Options.OutputType.rawValue) executable with xctestify.")
//inject platform-specific flags
#if os(OSX)
switch(Platform.targetPlatform) {
case .OSX:
compileOptions.append(contentsOf: ["-F", "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/Library/Frameworks/"])
linkOptions.append(contentsOf: ["-F", "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/Library/Frameworks/", "-target", "x86_64-apple-macosx10.11", "-Xlinker", "-rpath", "-Xlinker", "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/Library/Frameworks/", "-Xlinker", "-bundle"])
#endif
case .Linux:
break
}
}
if task[Options.XCTestStrict.rawValue]?.bool == true {
#if os(OSX)
//inject XCTestCaseProvider.swift
var xcTestCaseProviderPath = "/tmp/XXXXXXX"
var template = xcTestCaseProviderPath.cString(using: NSUTF8StringEncoding)!
xcTestCaseProviderPath = String(cString: mkdtemp(&template), encoding: NSUTF8StringEncoding)!
xcTestCaseProviderPath += "/XCTestCaseProvider.swift"
try! ATllbuild.xcTestCaseProvider.write(toFile: xcTestCaseProviderPath, atomically: false, encoding: NSUTF8StringEncoding)
sources.append(xcTestCaseProviderPath)
#endif
switch Platform.targetPlatform {
case .OSX:
//inject XCTestCaseProvider.swift
var xcTestCaseProviderPath = "/tmp/XXXXXXX"
var template = xcTestCaseProviderPath.cString(using: NSUTF8StringEncoding)!
xcTestCaseProviderPath = String(cString: mkdtemp(&template), encoding: NSUTF8StringEncoding)!
xcTestCaseProviderPath += "/XCTestCaseProvider.swift"
try! ATllbuild.xcTestCaseProvider.write(toFile: xcTestCaseProviderPath, atomically: false, encoding: NSUTF8StringEncoding)
sources.append(xcTestCaseProviderPath)
case .Linux:
break
}
}
let moduleMap: ModuleMapType
if task[Options.ModuleMap.rawValue]?.string == "synthesized" {
......@@ -403,11 +411,16 @@ final class ATllbuild : Tool {
if task[Options.BootstrapOnly.rawValue]?.bool == true {
bootstrapOnly = true
//update the build platform to be the one passed on the CLI
Platform.buildPlatform = Platform.targetPlatform
}
else {
bootstrapOnly = false
}
///The next task will not be bootstrapped.
defer { Platform.buildPlatform = Platform.hostPlatform }
let sdk: Bool
if task[Options.LinkSDK.rawValue]?.bool == false {
sdk = false
......@@ -422,9 +435,9 @@ final class ATllbuild : Tool {
else {
llbuildyamlpath = workDirectory + "llbuild.yaml"
}
let swiftCPath: String
if let c = task[Options.SwiftCPath.rawValue]?.string {
print("Warning: \(Options.SwiftCPath.rawValue) is deprecated and will be removed in a future release of atbuild. Use --toolchain to specify a different toolchain, or --platform when bootstrapping to a different platform.")
swiftCPath = c
}
else {
......@@ -460,7 +473,7 @@ final class ATllbuild : Tool {
case .StaticLibrary:
try! copyByOverwriting(fromPath: "\(workDirectory)/products/\(name).a", toPath: "bin/\(name).a")
case .DynamicLibrary:
try! copyByOverwriting(fromPath: "\(workDirectory)/products/\(name)\(DynamicLibraryExtension)", toPath: "bin/\(name)\(DynamicLibraryExtension)")
try! copyByOverwriting(fromPath: "\(workDirectory)/products/\(name)\(Platform.targetPlatform.dynamicLibraryExtension)", toPath: "bin/\(name)\(Platform.targetPlatform.dynamicLibraryExtension)")
}
switch moduleMap {
case .None:
......
;; Copyright (c) 2016 Anarchy Tools Contributors.
;;
;; Licensed under the Apache License, Version 2.0 (the "License");
;; you may not use this file except in compliance with the License.
;; You may obtain a copy of the License at
;;
;; http:;;www.apache.org/licenses/LICENSE-2.0
;;
;; Unless required by applicable law or agreed to in writing, software
;; distributed under the License is distributed on an "AS IS" BASIS,
;; WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
;; See the License for the specific language governing permissions and
;; limitations under the License.
(package
:name "platforms"
:tasks {
:build {
:tool "atllbuild"
:sources ["main.swift"]
:name "platforms"
:output-type "executable"
:publish-product true
:overlays {
:atbuild.platform.osx {
:compile-options ["-D" "OSX"]
}
:atbuild.platform.linux {
:compile-options ["-D" "LINUX"]
}
:bootstrap-only {
:bootstrap-only true
:llbuildyaml "bin/bootstrap-platform.yaml"
}
}
}
:check {
:dependencies ["build"]
:tool "shell"
:script "bin/platforms"
}
}
)
client:
name: swift-build
tools: {}
targets:
"": [<atllbuild>]
atllbuild: [<atllbuild>]
commands:
<atllbuild-swiftc>:
tool: swift-compiler
executable: "//usr/local/bin/swiftc"
inputs: ["main.swift"]
sources: ["main.swift"]
objects: [".atllbuild/objects/main.swift.o"]
outputs: ["<atllbuild-swiftc>", ".atllbuild/objects/main.swift.o"]
module-name: platforms
module-output-path: .atllbuild/products/platforms.swiftmodule
temps-path: .atllbuild//llbuildtmp
other-args: ["-j8", "-D", "ATBUILD", "-I", ".atllbuild/products/", "-D", "LINUX"]
<atllbuild>:
tool: shell
inputs: ["<atllbuild-swiftc>", ".atllbuild/objects/main.swift.o"]
outputs: ["<atllbuild>", ".atllbuild/products/platforms"]
args: ["//usr/local/bin/swiftc", "-o", ".atllbuild/products/platforms", ".atllbuild/objects/main.swift.o"]
description: Linking executable .atllbuild/products/platforms
client:
name: swift-build
tools: {}
targets:
"": [<atllbuild>]
atllbuild: [<atllbuild>]
commands:
<atllbuild-swiftc>:
tool: swift-compiler
executable: "/Library/Developer/Toolchains/swift-latest.xctoolchain/usr/bin/swiftc"
inputs: ["main.swift"]
sources: ["main.swift"]
objects: [".atllbuild/objects/main.swift.o"]
outputs: ["<atllbuild-swiftc>", ".atllbuild/objects/main.swift.o"]
module-name: platforms
module-output-path: .atllbuild/products/platforms.swiftmodule
temps-path: .atllbuild//llbuildtmp
other-args: ["-j8", "-D", "ATBUILD", "-I", ".atllbuild/products/", "-sdk", "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk", "-D", "OSX"]
<atllbuild>:
tool: shell
inputs: ["<atllbuild-swiftc>", ".atllbuild/objects/main.swift.o"]
outputs: ["<atllbuild>", ".atllbuild/products/platforms"]
args: ["/Library/Developer/Toolchains/swift-latest.xctoolchain/usr/bin/swiftc", "-o", ".atllbuild/products/platforms", ".atllbuild/objects/main.swift.o"]
description: Linking executable .atllbuild/products/platforms
#if OSX
print("Hello from OSX!")
#elseif LINUX
print("Hello from LINUX!")
#endif
\ No newline at end of file
......@@ -10,6 +10,36 @@ pwd
echo "****************SELF-HOSTING TEST**************"
$ATBUILD atbuild
echo "****************PLATFORMS TEST**************"
cd $DIR/tests/fixtures/platforms
UNAME=`uname`
$ATBUILD check 2&> /tmp/platforms.txt
if [ "$UNAME" == "Darwin" ]; then
STR="Hello from OSX!"
else
STR="Hello from LINUX!"
fi
if ! grep "$STR" /tmp/platforms.txt; then
cat /tmp/platforms.txt
echo "Did not find platform print in platform test"
exit 1
fi
#check bootstrapping case
$ATBUILD build --use-overlay bootstrap-only --platform linux
if ! cmp --silent bin/bootstrap-platform.yaml known-linux-bootstrap.yaml; then
echo "Linux bootstrap was unexpected"
exit 1
fi
$ATBUILD build --use-overlay bootstrap-only --platform osx
if ! cmp --silent bin/bootstrap-platform.yaml known-osx-bootstrap.yaml; then
echo "OSX bootstrap was unexpected"
exit 1
fi
echo "****************XCODE TOOLCHAIN TEST**************"
if [ -e "/Applications/Xcode.app" ]; then
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment