1. 13 Nov, 2016 1 commit
  2. 03 Nov, 2016 1 commit
  3. 09 Sep, 2016 1 commit
    • Drew's avatar
      Support Xcode 8 GM · a4d211ee
      Drew authored
      * Drop test coverage for Xcode 7 since it's likely not installed
      * We now pull paths from xcode-select when we don't specify a toolchain which is saner than using hardcoded paths
      * We now more robustly test if a given toolchain is from xcode 7 or not rather than rely on the filename
      * Bootstrap for Xcode 8 installed to Xcode.app
      a4d211ee
  4. 29 Aug, 2016 2 commits
  5. 18 Aug, 2016 1 commit
  6. 14 Aug, 2016 1 commit
  7. 08 Aug, 2016 1 commit
  8. 02 Aug, 2016 1 commit
    • Drew's avatar
      Update to swift preview 3 · e4524c67
      Drew authored
      * We can't use system anymore in preview 3.  This introduces a lot of issues around envrionment variables, which can't be listed in swift :-(.  So we no longer inherit environment variables, we only set the ones we set.
          * We do pass on PWD and PATH, because otherwise that would be terrible
      * Toolchain is now a part of platform config instead of something we pass around by hand.
      * We now detect xcode 7 / 8 depending on whether we're using a toolchain installed to Xcode.app or Xcode-beta.app.  That's still not right, but fuck it.
      e4524c67
  9. 08 Jul, 2016 3 commits
    • Drew's avatar
      Add support for C language to atllbuild · e2fa4f07
      Drew authored
      This PR lets you mix .swift, .h, and .c files all in the same atllbuild task.  It works a lot like Xcode's behavior, if you've used that.
      
      \# Rationale
      
      I feel the need to defend this feature, since I have been previously on the record as saying "the entire value is debatable" (https://www.mail-archive.com/swift-evolution@swift.org/msg01829.html).
      
      There are 5 cases where I think it makes sense to add a little C to your Swift project:
      
      * To use the odd feature Swift doesn't support.  Recently, I needed to call a variadic C function; Swift cannot call them, C is our only hope
      * To work around a Swift compiler bug.  Several of my projects have this case.
      * To repackage an existing Xcode project where somebody used C in it.  I have not investigated and don't want to investigate whether that somebody was sane or insane, but we should at least be able to build their project.
      * To include headers from a system C library.  SwiftPM tries to solve this with module maps, however it doesn't hide the implementation details https://bugs.swift.org/browse/SR-655.  This feature can actually hide them with a few different methods discussed below, which is a clear win.
      * To write Swift bindings for a C library.  This generally involves a little C glue code (such as using a header or something), and for reasons that will become clear, using our C support is better than previous approaches at that problem.
      
      \# Rationale-NOT:
      
      Additional rationale:
      
      * SwiftPM will probably add this eventually
      * per #113, we should be a superset of their functionality
      
      I would like to be very clear about my goals:
      
      * This is really only designed for the case of "need a little C in your Swift project", not anything larger
      * This is not a replacement for e.g. GNU Make or a general-purpose C buildsystem, nor will it become one
      * The preferred mechanism for building a real C library is shelling out to your real C buildsystem
      * Nobody should be repackaging their established C libraries as atllbuild tasks.  atllbuild is designed to build Swift projects, not C projects.
      
      \# Design
      
      * You can now specify `.c` and `.h` files in the sources for atllbuild tasks
          * Also `**.c` and `**.h` just like `**.swift`
          * Like `.swift`, no files are scanned by default, everything is explicit
      * Adding `.c` files causes them to be compiled and linked into the atllbuild task just like swift files
      * Adding `.h` files exposes declarations to Swift.  It works much like a bridging header; put stuff in header files and then Swift code will see it.
          * Your `.h`s can import other `.h`s (from the system, or anywhere else) and you otherwise have access to the complete C preprocessor
      * New atllbuild setting `c-compile-options` specifies compile options for C files.  `compile-options` is ignored for C.
      * C files work as you'd expect, including support for things like configurations, optimization, atbuild preprocessor macros, etc.
      
      \# Linking
      
      The standard `link-options` sets link options for both C and Swift; since they are linked into the same library there is no individual control.  So if you want to link your C (and Swift) code against curl, you could say `:link-options ["-lcurl"]` for example.
      
      The problem with this approach is that everybody who depends on you might also need `-lcurl`.  Traditionally we've solved this with overlays that we expose to callers.
      
      SwiftPM avoids this problem by requiring everyone to create e.g. `CCurl` everywhere: https://github.com/apple/swift-package-manager/blob/master/Documentation/SystemModules.md
      
      And in fact people do: https://github.com/IBM-Swift/CCurl
      
      The problem is now you have to import `CCurl` everywhere (even in files that don't directly use it).  See generally, https://bugs.swift.org/browse/SR-655, https://gist.github.com/briancroom/5d0f1b966fa9ef0ae4950e97f9d76f77
      
      Here is the cool part though.  This PR adds a new option `:module-map-link ["curl"]`.  That will inject a link directive into both the module map we use at buildtime and the one we export e.g. into an atbin.
      
      Emitting that link directive has the effect of injecting `:link-options ["-lcurl"]`.  However, it will *also* inject that link option into any Swift module that imports this one.  The result is that downstream no longer needs to add `:link-options ["-lcurl"]` anymore.
      
      Additonally, since we achieve this in a single module, there is no `CCurl` to import anymore.  The details of linking to the C library are more effectively hidden.
      
      For these reasons, I believe using the C support in this PR is way more effective for writing bindings than any other solution.
      
      \# Known issues
      
      * Using `.h` in `sources` requires a synthesized module map
      * Using `.c` in `sources` is not supported for bitcode
      * Using `module-map-link` requires the module map to be distributed for the link to take effect on downstream; we recommend `packageatbin` for packing build products
      * Currently, swift functions are not "visible" to C code (like they are visible to ObjC from Xcode) although presumably if you had a function, knew its calling convention, and knew its c-name, you could totally call it from C.
      e2fa4f07
    • Drew's avatar
      Supress "unknown option module-map" · 8b4959e3
      Drew authored
      8b4959e3
    • Drew's avatar
      Fix Xcode 8 beta 2 · d1b8e7d1
      Drew authored
      d1b8e7d1
  10. 30 Jun, 2016 3 commits
    • Drew's avatar
      Un-deprecate WMO setting · 3c62219a
      Drew authored
      We can't enable WMO for Release on Linux yet, see https://github.com/AnarchyTools/atbuild/issues/110
      3c62219a
    • Drew's avatar
      Add actual effects to the configurations · 8f797026
      Drew authored
      This extends #104 by adding actual effects to the configurations rather than have them be no-ops.
      
      Effects include:
      
      * debug instrumentation (new in this PR), for emitting `-g` (see #73 for an obvious extension)
      * optimization control / WMO control
      * compression level (faster debug atbins)
      * test instrumentation (`-enable-testing`)
      * `#if ATBUILD_RELEASE` etc. from Swift code
      
      There are some deprecations associated with this PR:
      
      * `whole-module-optimization` atllbuild option is now deprecated; use `--configuration release` instead.  There currently is no plan to control these separately, use `--configuration plain` + `:compileOptions ["-O"]` to get optimization without WMO.  Or open a bug to complain about this change.
      * `magic` atllbuild option is now deprecated; to opt out of magic use `--configuration none` instead.
      
      Doc PR to follow.
      
      In addition, CI is now updated to produce release (optimized) builds for atbuild, which significantly optimizes atbuild performance.
      8f797026
    • Drew's avatar
      Remove atllbuild option swiftc-path · 3798f21b
      Drew authored
      This option was deprecated in atbuild 0.9.0.  Use `--toolchain` on the CLI instead.
      
      Users should have had enough time to migrate at this point.
      3798f21b
  11. 21 Jun, 2016 1 commit
  12. 07 Jun, 2016 1 commit
  13. 27 May, 2016 1 commit
  14. 21 May, 2016 1 commit
    • Drew's avatar
      Implement configurations · b2ba7bbe
      Drew authored
      This implements, more or less, the scaffolding described in #36.  This commit does not actually make tools behave differently in any configuration (so this is mostly a placebo) but after this, tools can adjust their behavior.
      
      Notable additions/departures from the original proposal include:
      
      * Support for "custom" configurations outside the built-in set
      * The addition of `test` and `bench` as built-in configurations, since 2/2 developers use them
      * Internal API has new "helper" methods for common "tool questions" (should we optimize, are we testing, etc.)  Ideally, custom configurations could indicate their own values for these questions, although that's outside the scope of this patch.
      
      Doc PR to follow.
      b2ba7bbe
  15. 18 May, 2016 1 commit
    • Drew's avatar
      Fix WMO · 1aba8e49
      Drew authored
      We previously used a (pretty bad) hack for WMO.  This resulted in issues like #92.
      
      Upstream now has proper support for WMO (see generally, https://github.com/apple/swift-llbuild/pull/28, https://bugs.swift.org/browse/SR-881).
      
      We now use the upstream feature to handle this case.  We also add -num-threads support, which upstream recently added.
      
      Note that our implementation now only works for swift-DEVELOPMENT-SNAPSHOT-2016-05-09-a and above.
      
      Resolve #92
      1aba8e49
  16. 12 May, 2016 4 commits
  17. 11 May, 2016 3 commits
    • Drew's avatar
      Add ATBUILD_BIN_PATH · 5e877eb5
      Drew authored
      This adds a new environment variable to point to the bin path.  This is useful for custom tool packagers.
      
      We also unified the implementation of shell and custom tool environments in the new Shell.environment function.
      
      There are some minor changes to the custom tool environment that come along with this change, such as running in the directory of the imported package.  To my knowledge, I'm the only one who will notice.
      5e877eb5
    • Drew's avatar
      6c8802e0
    • Drew's avatar
      Modify filenames for compressed atbin · 9a26e865
      Drew authored
      new format is {name}-{version?}-{target}.atbin.tar.xz
      9a26e865
  18. 10 May, 2016 4 commits
    • Drew's avatar
      Support atbin compression · 3982ceb4
      Drew authored
      3982ceb4
    • Drew's avatar
      Add only-platforms · 658955ba
      Drew authored
      A task-level mechanism to skip tasks on some platforms
      658955ba
    • Drew's avatar
      Set environment variable $ATBUILD_PACKAGE_VERSION · 36f4cbef
      Drew authored
      36f4cbef
    • Drew's avatar
      Add executable-name option to atllbuild · 8af5d3c8
      Drew authored
      We add an executable-name option to atllbuild, allowing the use of "non-module-safe" names for executables.  This includes e.g. hyphens, which are a legal executable name but not a legal module-name.
      
      Resolve #27.  This resolution was chosen (over name/module-name) because the module-name is used in several places (such as Frameworks for example) and the executable case seems to be the odd one out at present.
      8af5d3c8
  19. 05 May, 2016 1 commit
  20. 28 Apr, 2016 2 commits
    • Drew's avatar
      Change name to `compiled.atpkg` · 393fb733
      Drew authored
      393fb733
    • Drew's avatar
      Add atbin support · 1c22e037
      Drew authored
      atbin is a proposed binary interchange format for atbuild and the
      broader AT ecosystem.
      
      atbuild has a weakly standardized input format: `build.atpkg`.  But what
      does atbuild, um, actually build?  What's the output format?
      
      There is no weak standard here, or even a convention.  It may build an
      executable, a static library, or a dynamic one, or even a framework; it
      may emit swiftmodule and swiftdoc files, or not.  A modulemap may or may
      not be part of the build products and clients may or may not need it in
      their search paths.
      
      The uncertainty here complicates interoperability.  atpm should download
      binary libraries, but what actually lives in that tarball?  Some random
      dylibs we found in `.atllbuild/products`?
      
      How do we build libraries for "fat" archs (like iOS, with 4 sub-archs)?
      How would we write an `atinstall` tool that installs/updates
      atpm/atbuild (or makes homebrew packages out of them)?
      
      atbin proposes to answer these questions, providing a simple, portable,
      hackable binary interchange format for all platforms and all AT
      projects.
      
      An `atbin` is a folder that ends in `.atbin`.  It contains, at least, a
      `built.atpkg` file.
      
      `built.atpkg` is clojure syntax identical to the more familiar
      `build.atpkg`.  You can include tasks or w/e in there, although why
      would want to, I'm not totally sure (this is Anarchy Tools though,
      knock yourself out.)  However, the important bit is this:
      
      ```clojure
      (package
          :name "foo"
          :payload "libFoo.a"
          :platforms ["ios-x86_64" "ios-i386"]
          :type "static-library"
      )
      ```
      
      (Other fields could also be present, this is not a complete enumeration)
      
      This `.atbin` folder will then contain:
      
      * `libFoo.a`, a fat library for the indicated platforms
      * (optional) a `ios-x86_64.swiftmodule` and `ios-i386.swiftmodule` file
      * (optional) a `ios-x86_64.swiftdoc` and `ios-i386.swiftdoc` file
      * (optional) a `module.modulemap` file
      
      You can, of course, build an `.atbin` by hand from existing binaries you
      found lying around your disk.  And we may eventually ship an `atbin`
      packager for Xcode or SwiftPM projects.
      
      However more practically, we introduce a new tool, `packageatbin`, which
      packages an `atbin` payload from atllbuild.
      
      ```clojure
      :package {
         :tool "packageatbin"
      
         ;; Generate a mypayload.atbin
         :name "mypayload"
      
         ;; When used with the new --platform ios, will build a fat binary for all iOS platforms.
         ;; Alternatively specific platforms can be listed here
         :platforms ["all"]
      
         ;; The atllbuild task to package.
         ;; Special logic will re-run this task for each platform and merge the resulting output.
         :atllbuild-task "myatllbuildtask"
      }
      ```
      
      The obvious application is as an interchange format for prebuilt
      `atllbuild` dependencies.  Presently, `atllbuild` can link with the
      output of any dependent atllbuild task, but if a library wasn't produced
      by a dependent task as part of the current build (but was say produced
      on a different computer a month ago) there's no "obvious" way to link to
      it.  This PR does not actually include any of that tooling, but it would
      be a straightforward extension of this work.
      
      An second application is the building of fat files.  Currently, there is
      no mechanism to build a "fat" library or binary in atbuild, or even to
      specify that we want one.  Under this PR, we can do it.
      
      A third application is a distribution format for executables.  If an
      `.atbin` contains an `executable`, `atpm` (or hypothetical `atinstall`)
      could install/update/administrate that executable similar to `homebrew`
      or `apt`, and keep all your buildtools (or other random crap) up to
      date.  We would need to extend this with version fields and whatnot, but
      again, it's straightforward.
      
      An fourth application, and my real motivation, is as an intermediate
      binary representation.  An `atbin` can be "downcast" with another tool
      to a platform-native format like `deb`, `bottle`, or `Framework`.  This
      would allow us to cut debs, rpms, and framework releases with
      appropriate AT tools.
      
      One alternative is to adopt an existing "standard", like Framework, for
      this purpose.  Indeed, `atbuild` currently produces frameworks on OSX.
      
      There are some complexities of extending frameworks to this case.  For
      one, the Framework implementation is warty and involves a lot of
      symlinks and things like codesigning.  We don't currently maintain that
      code for Linux hosts, nor is the standard especially sensible for Linux,
      as it relies on plists and choices basically unpopular on that platform.
      
      For another, frameworks are not really built to house static library or
      executable payloads, which are important to atbuild.  There are air-
      quote "obvious" ways to extend to nontraditional payloads, but IMO this
      is more confusing than it is helpful.  An explicit step to "cast down"
      your atbin to a framework lets us check that your framework will
      actually make sense to the likes of Xcode.
      
      For a third, it's unclear what platform some random Framework is built
      for, and what architectures it supports.  You can find out by scripting
      out to platform-specific tools, but it's not portable.
      
      Another alternative is to support multiple payloads/libraries in a
      single atbin, "one atbin to rule them all".  However I don't see what we
      accomplish there that we don't accomplish with multiple atbins, except
      specification complexity.  So let's not do that, at least not initially.
      
      `packageatbin` is included in core primarily because it needs tight,
      source-level integration with atllbuild.  In addition to peeking at the
      atllbuild options it needs to run the atllbuild task several times in
      order to produce fat binaries, which means it has to work around the
      usual dependency pruning logic.  For that reason it can't be sensibly
      implemented via the current custom tool API.
      1c22e037
  21. 25 Apr, 2016 2 commits
  22. 24 Apr, 2016 1 commit
    • Drew's avatar
      Add iOS support · 656d6bb3
      Drew authored
      This commit adds support for static libraries, dynamic libraries, and
      executables compiled for iOS.
      
      FAQ:
      
      Q: How do I build them?
      
      Use the new `--platform` strings:
      
      * `ios-x86_64`
      * `ios-i386`
      * `ios-arm64`
      * `ios-armv7`
      
      Q: What if I want to build for more than one architecture?
      
      Coming Soon
      
      Q: What is an iOS "executable", anyway?
      
      No idea, but it works!
      
      Q: What is not yet supported?
      
      - [ ] XCTest
      - [ ] Deploying or running iOS build products
      - [ ] Frameworks
      - [ ] Code signing
      - [ ] Compiling for iOS on Linux.  Believe it or not, I think this
            is actually possible for some programs, but I have no use for it
      656d6bb3
  23. 22 Apr, 2016 3 commits