Design and architecture: Special Dumpster Fire Unit

Design and architecture: Special Dumpster Fire Unit

This is a keynote highlight from the O’Reilly Software Architecture Conference in New York 2019. Watch the full version of this keynote on the O’Reilly online learning platform.

You can also see other highlights from the event.

Article image: Chess cube

(source: Pixabay).

Roaming free: The power of reading beyond your field

Roaming free: The power of reading beyond your field

This is a keynote highlight from the O’Reilly Software Architecture Conference in New York 2019. Watch the full version of this keynote on the O’Reilly online learning platform.

You can also see other highlights from the event.

Article image: Compass

(source: Pixabay).

Design after Agile: How to succeed by trying less

Design after Agile: How to succeed by trying less

This is a keynote highlight from the O’Reilly Software Architecture Conference in New York 2019. Watch the full version of this keynote on the O’Reilly online learning platform.

You can also see other highlights from the event.

Four short links: 6 February 2019

Four short links: 6 February 2019

Four short links
  1. Flowbladea multitrack non-linear video editor released under GPL3 license.
  2. Automatically Assembling Textbooks from WikipediaAdamti and co have a plan for determining the utility of their approach. They plan to produce a range of Wikibooks on subjects not yet covered by human-generated books. They will then monitor the page views and edits to these books to see how popular they become and how heavily they are edited, compared with human-generated books.
  3. Amazon Knows What You Buy. And It’s Building a Big Ad Business From It (NYT) — I’m sure nothing bad can happen from this.
  4. Firefox 66 to Block Automatically Playing Audible Video and Audio (Mozilla) — user-friendly behavior ftw.
Article image: Four short links

170+ live online training courses opened for March and April

170+ live online training courses opened for March and April

Computer and other gear

(source: Skitterphoto via Pixabay)

Learn new topics and refine your skills with more than 170 new live online training courses we opened up for March and April on the O’Reilly online learning platform.

AI and machine learning

Spotlight on Innovation: Succeeding with Machine Learning with Alex Jaimes, February 13

Hands-On Adversarial Machine Learning, February 25

Probabilistic Modeling With TensorFlow Probability, February 27

Deep Learning Fundamentals, March 5

An Introduction to Amazon Machine Learning on AWS, March 6-7

Natural Language Processing (NLP) from Scratch, March 11

Deep Reinforcement Learning, March 12

Sentiment Analysis for Chatbots in Python, March 13

Hands-on Machine Learning with Python: Classification and Regression, March 13

TensorFlow Extended: Data Validation and Transform, March 14

Hands-On Machine Learning with Python: Clustering, Dimension Reduction, and Time Series Analysis, March 14

Building a Robust Machine Learning Pipeline, March 14-15

Machine Learning in Practice, March 19

TensorFlow Extended: Model Build, Analysis, and Serving, March 20

Artificial Intelligence: An Overview of AI and Machine Learning, March 20

Machine Learning for IoT , March 20

Next Generation Decision Making: Pragmatic Artificial Intelligence, March 20-21

Getting Started with Machine Learning, March 21

Artificial Intelligence for Robotics, March 21-22

Beginning Machine Learning with PyTorch, March 25

Artificial Intelligence: Real-World Applications, March 28

Active Learning, April 9

Hands On Adversarial Machine Learning, April 11

Practical Deep Learning with PyTorch, April 11-12

Blockchain

Introducing Blockchain, March 8

Building Smart Contracts on the Blockchain, March 21-22

IBM Blockchain Platform as a Service, March 25-26

Understanding Hyperledger Fabric Blockchain, March 28-29

Blockchain for Enterprise, April 1

Business

Innovative Teams, March 11

Fundamentals of Cognitive Biases, March 11

Artificial Intelligence: AI For Business, March 12

Business Strategy Fundamentals, March 13

The Power of Lean in Software Projects: Less Wasted Effort and More Product Results, March 14

Leadership Communication Skills for Managers, March 14

Emotional Intelligence in the Workplace, March 14

Thinking Like a Manager, March 14

Tools for the Digital Transformation, March 14-15

Introduction to Delegation Skills, March 21

Negotiation Fundamentals, March 22

Introduction to Critical Thinking, March 26

Your First 30 Days as a Manager, April 2

How to Give Great Presentations, April 5

Introduction to Strategic Thinking Skills, April 8

Data science and data tools

Business Data Analytics Using Python, February 27

Hands-on Introduction to Apache Hadoop and Spark Programming, March 5-6

Designing and Implementing Big Data Solutions with Azure, March 11-12

Time Series Forecasting, March 14

Cleaning Data at Scale, March 19

Practical Data Cleaning with Python, March 20-21

Building Distributed Pipelines for Data Science Using Kafka, Spark, and Cassandra , April 8-10

Real-Time Data Foundations: Kafka, April 9

Real-Time Data Foundations: Spark, April 10

Building Data APIs with GraphQL, April 11

Design and product management

From User Experience Designer to Digital Product Designer, March 1

Mastering UX Mapping, March 7-8

Writing User Stories, March 13

Product Roadmaps from the Ground Up, April 3

Programming

Design Patterns Boot Camp, February 19-20

Discovering Modern Java, March 1

Beginner’s Guide to Writing AWS Lambda Functions in Python, March 1

Building APIs with Django REST Framework, March 4

SQL for Any IT Professional, March 4

Spring Boot and Kotlin, March 5

Programming with Java Lambdas and Streams, March 5

Bootiful Testing, March 6

Learning Python 3 by Example, March 7

Getting Started with OpenShift, March 8

Setting Up Scala Projects, March 11

Getting Started with Pandas, March 11

Getting Started with Python 3, March 11-12

Java Full Throttle with Paul Deitel: A One-Day, Code-Intensive Java Standard Edition Presentation, March 12

Mastering Pandas, March 12

Scalable Concurrency with the Java Executor Framework, March 12

Getting Started with Python’s Pytest, March 13

Python Programming Fundamentals, March 13

Mastering Python’s Pytest, March 14

Kotlin Fundamentals, March 14

Quantitative Trading with Python, March 14

Advanced TDD (Test-Driven Development), March 15

Introduction to Python Programming, March 15

Bash Shell Scripting in 4 Hours, March 18

Java Testing with Mockito and the Hamcrest Matchers, March 19

Scala Core Programming: Methods, Classes Traits, March 19

Ansible in 4 Hours, March 19

Getting Started with PHP and MySQL , March 20

Mastering the Basics of Relational SQL Querying, March 20-21

Reactive Spring and Spring Boot, March 21

Automating with Ansible, March 22

Scala Core Programming: Sealed Traits, Collections, and Functions, March 25

Mastering SELinux, March 25

Intermediate Git, March 25

Scalable Programming with Java 8 Parallel Streams, March 27

Design Patterns Boot Camp, March 27-28

Mastering C# 8.0 and .NET Core 3.0, March 27-28

Rethinking REST: A Hands-On Guide to GraphQL and Queryable APIs, March 28

C# Programming: A Hands-On Guide, March 28

Web Application Programming in C# and ASP.NET Core with MVC and Entity Framework, March 28-29

Introduction to JavaScript Programming, April 2-3

Visualization in Python with Matplotlib, April 8

Python for Finance, April 8-9

Practical MQTT for the Internet of Things, April 8-9

Getting Started with Pandas, April 9

Getting Started with Python 3, April 9-10

Getting Started with React.js, April 10

What’s New In Java, April 11

Fundamentals of Rust, April 11-12

Security

CompTIA PenTest+ Crash Course, March 5-6

Start Your Security Certification Career Today, March 8

Protecting Data Privacy in a Machine Learning World, March 11-12

Certified Ethical Hacker (CEH) Crash Course, March 12-13

CompTIA Security+ SY0-501 Crash Course, March 18-19

Intense Introduction to Hacking Web Applications, March 19

Cyber Security Fundamentals, March 26-27

CISSP Crash Course, March 26-27

CISSP Certification Practice Questions and Exam Strategies, March 27

AWS Certified Security – Specialty Crash Course, March 27-28

Systems engineering and operations

Software Architecture by Example, February 21

Red Hat Certified System Administrator (RHCSA) Crash Course, March 4-7

Creating Serverless APIs with AWS Lambda and API Gateway, March 5

Amazon Web Services (AWS): Up and Running, March 6

Docker Compose, March 6

Microservice Collaboration, March 7

Docker CI/CD, March 7

OpenStack for Cloud Architects, March 7-8

Red Hat RHEL 8 New Features, March 11

From Developer to Software Architect, March 11-12

Google Cloud Certified Associate Cloud Engineer Crash Course, March 11-12

AWS Certified Solutions Architect Associate Crash Course, March 11-12

9 Steps to Awesome with Kubernetes, March 12

IP Subnetting from Beginning to Mastery, March 12-13

Istio on Kubernetes: Enter the Service Mesh, March 14

How the Internet Really Works, March 15

Kubernetes Serverless with Knative, March 15

AWS Advanced Security with Config, GuardDuty, and Macie, March 18

Software Architecture by Example, March 18

Amazon Web Services: AWS Managed Services, March 18-19

Practical Kubernetes, March 18-19

AWS Certified SysOps Administrator (Associate) Crash Course, March 18-19

CCNA Routing and Switching 200-125 Crash Course, March 18-22

Managing Containers on Linux, March 19

Docker Images, March 19

Docker: Up and Running, March 19-20

Docker Containers, March 20

Implementing Evolutionary Architectures, March 20-21

Kubernetes in 4 Hours, March 21

AWS Security Fundamentals, March 21

Deploying Container-Based Microservices on AWS, March 21-22

Google Cloud Platform (GCP) for AWS Professionals, March 22

Architecture for Continuous Delivery , March 25

Docker for JVM projects, March 25

Implementing Azure for Enterprises, March 25-26

Building and Managing Kubernetes Applications, March 26

Cloud Computing Governance, March 26

Getting Started with Amazon Web Services (AWS), March 26-27

Microservices Caching Strategies, March 27

Cloud Complexity Management, March 28

Comparing Service-Based Architectures, March 28

Network DevOps, March 29

API Driven Architecture with Swagger and API Blueprint, March 29

Software Architecture for Developers, April 1

Implementing and Troubleshooting TCP/IP, April 2

Amazon Web Services (AWS) Technical Essentials, April 2

Building Applications with Apache Cassandra, April 3-4

Introduction to Kubernetes, April 3-4

CCNA Routing and Switching Crash Course, April 4-5

Architecting Secure IoT Applications with Azure Sphere, April 4-5

AWS Design Fundamentals, April 9-10

Microservices Architecture and Design, April 9-10

Practical Docker, April 10

Automation with AWS Serverless Technologies, April 10

From the trenches: An interview with Mark Richards

From the trenches: An interview with Mark Richards

This is a keynote highlight from the O’Reilly Software Architecture Conference in New York 2019. Watch the full version of this keynote on the O’Reilly online learning platform.

You can also see other highlights from the event.

Highlights from the O’Reilly Software Architecture Conference in New York 2019

Highlights from the O’Reilly Software Architecture Conference in New York 2019

New York City Skyline – O’Reilly Software Architecture Conference NY 2019

People from across the software architecture world are coming together in New York for the O’Reilly Software Architecture Conference. Below you’ll find links to highlights from the event.

The future of cloud-native programming

Tamar Eilam offers an overview of cloud-native programming and outlines a path toward the unification of the cloud programming model.

Career advice for architects

Trisha Gee shares lessons she learned the hard way while managing her career as a developer, lead, and technical advocate.

From the trenches: An interview with Mark Richards

Neal Ford talks with Mark Richards about his career path and his work as a software architect.

Article image: New York City Skyline – O’Reilly Software Architecture Conference NY 2019

The future of cloud-native programming

The future of cloud-native programming

This is a keynote from the O’Reilly Software Architecture Conference in New York 2019. See other highlights from the event.

This keynote was sponsored by IBM.

Article image: Cloud

(source: Pixabay).

Four short links: 5 February 2019

Four short links: 5 February 2019

Four short links
  1. The Best Way to Predict the Future is to Create It. But Is It Already Too Late? (Alan Kay) — Virtually everybody in the computing science has almost no sense of human history and context of where we are and where we are going. So, I think of much of the stuff that has been done as inverse vandalism. Inverse vandalism is making things just because you can. Every sentence is a cracker. (via Daniel G. Siegel)
  2. Trying to Make Powerful, Low-cost LIDAR (Ars Technica) — a good intro to the tech and competition in the space.
  3. Guidelines for Human-AI Interaction — Microsoft paper on design challenges in “smart” apps.
  4. MIT Hacker Tools — lectures on the Unix tools that command-line natives use.
Article image: Four short links

3 emerging trends tech leaders should watch

3 emerging trends tech leaders should watch

Radar dish at dusk

(source: michaelqiao13591 via Pixabay)

Keeping up with technology can be a daunting task for tech leaders. Each year, to make the task a little easier, we analyze behavior on the O’Reilly online learning platform, using the platform as a massive sensor that yields insights and identifies areas tech leaders should pay attention to, explore, and learn.

Our analysis includes the top search terms and the topics that garner the most usage on our learning platform.[1] This combination of search and usage data provides a holistic view; search data shows the areas where subscribers are exploring, and usage identifies topics where they’re actively engaged.

The signals from the O’Reilly online learning platform reveal:

  • Strong growth in cloud topics and Kubernetes, as well as interest in containers and decomposition (microservices), points toward the rise of a “Next Architecture.”
  • Interest in blockchain, which we first noted in 2017, continues. While the full potential of the blockchain gets sorted out, consider that if you’re not investigating blockchain, someone you compete with is.
  • Python, Java, and JavaScript—the “big three” languages on our learning platform—continue to dominate usage year after year. In addition, Rust and Go showed growing interest on the platform, suggesting that organizations are using languages that emphasize developer productivity while also embracing languages that tilt the balance toward performance and scaling.
Figure 1. The top search terms on the O’Reilly online learning platform in 2018 (left) and the rate of change for each term (right)
Figure 2. Topics on the O’Reilly online learning platform with the most usage in 2018 (left) and the rate of change for each topic (right)

The signs of a Next Architecture

The growth we’ve seen on our online learning platform in cloud topics, in orchestration and container-related terms such as Kubernetes and Docker, and in microservices is part of a larger trend in how organizations plan, code, test, and deploy applications that we call the Next Architecture. This architecture allows fast, flexible deployment, feature flexibility, efficient use of programmer resources, and rapid adapting, including scaling, to unpredictable resource requirements. These are all goals businesses feel increasingly pressured to achieve to keep up with nimble competitors.

There are four aspects of the Next Architecture, each of which shows up in the platform’s search and usage data.

Figure 3. AWS, Kubernetes, Docker, and Microservices—each representing an important part of the Next Architecture—appear in the top search terms from the O’Reilly online learning platform

Decomposition

Organizations get a lot of benefits by breaking large and complex activities into small, loosely connected pieces. Through decomposition, these activities can be turned into standalone services that can be developed independently and linked together to create a more complex application. Microservices, the manifestation of decomposition, was the number 13 search term on our online learning platform in 2018.

Cloud

An organization needs the flexibility to adjust, scale, and innovate its digital presence—often across different time zones and geographies. The cloud supports these goals with compute instances that are fungible, coming and going as needed, and easy to replace automatically if failures are detected. The move toward decomposition (microservices) helps accelerate the trend toward the cloud by providing more impetus for quickly spinning up and managing services that support the need for dynamic, adaptable applications.

Cloud-related terms had a significant presence in the search and usage data. AWS, Amazon’s suite of cloud-based tools, was the number 4 search term, and it had 28% growth in year-over-year usage. Google Cloud (66% growth in usage over 2017) and Microsoft Azure (60% growth in usage) also increased. In addition, the topic “cloud migration” was up 40% in usage in 2018.

Containers

Containers provide a lightweight way to achieve the modularity favored by decomposition and the cloud. Docker, the number 7 search term in 2018, makes it easy to automate the deployment of the microservices that are created through decomposition.

Orchestration

The huge number of microservices running on containers—often in the hundreds or thousands—exceeds the capacity of humans to track and manage them. Orchestration tools, notably Kubernetes, fill the gap through rigorous specifications and automation. Kubernetes was the number 5 search term in 2018, jumping 11 spots, and usage growth was up a notable 160% year over year.

We’ll continue to explore the Next Architecture in the coming months.

Keep an eye on blockchain

Blockchain, which was one of the stars in our 2017 results, jumped seven spots in the top search terms (number 13), and it was up 36% in usage in 2018. Ethereum, a tool for implementing blockchains, was up 66% in year-over-year usage from a small base. Platform subscribers were likely exploring blockchain to assess its potential, developing an awareness of where blockchain may fit into their strategic plans or evaluating it as an existential threat, mostly in the areas of payments, supply chain logistics, and provenance.

Python, Java, and JavaScript continue their dominance

In 2018 we saw Python, Java, and JavaScript maintain the strong positions they’ve gained on our online learning platform over the years.

Python gets a boost, in part, from the increased interest in machine learning (ML). Many ML libraries, such as TensorFlow, are wrapped in Python libraries and promoted with Python interfaces. Ascendant ML tools also bolster interest in Python. For example, PyTorch, a library for computer vision and natural language processing, saw a 300% increase in year-over-year usage from a small base, and scikit-learn, another Python-based machine learning library, was up 39% in usage.

Many tools used in big data applications—notably the ones from the Apache Foundation, such as Spark and Kafka—feature Java interfaces. Thus, machine learning and big data may explain the popularity of both Python and Java. Java also remains a workhorse language for large-scale applications.

The JavaScript ecosystem of web frameworks and libraries saw less growth than Java and Python. However, usage trends show engagement with the popular JavaScript web frameworks. Angular was up 23% in usage, and React was up 39%, though search activity on both topics was flat. A third JavaScript framework, Vue, showed big usage growth, up 220% from a small base.

After JavaScript, one more language appears in our top searches. Go, the number 11 search term, jumped three spots in the top search results, and content usage was up 14%. Go sits conveniently between high-level languages, interpreted languages like Python, and low-level, fast systems-compiled languages like C. It combines the syntactic ease of the high-level languages with compiler-driven performance, good concurrency support, an active and growing developer community, and the full support of Google. When performance matters, or when an app or service written in a high-level language needs a performance boost, Go is (sorry for the pun) the go-to language for an increasing number of developers.

Finally, the fastest usage growth we saw for any language between 2017 and 2018 was for Rust (up 44%). Rust is a systems language with near-C performance, safe, efficient memory management, native concurrency support, and a modern syntax. Developers are increasingly finding Rust a good fit when performance is or becomes a priority.

Other findings

There are a few more items from the analysis that are worth calling out.

  • Machine learning (ML), the number 10 search term, has been a leader on our learning platform for more than a year, as we showed in last year’s trends. In 2018 we saw a change in the distribution of interest in ML topics within the search and usage results. There was less growth in exploratory topics and phrases like “machine learning” and “deep learning.” This was coupled with a shift toward more specific topics like “natural language processing” (up 22% in search and 11% in usage) and “reinforcement learning” (up 122% in search and 331% in usage from a small base). We attribute the shift to the maturation of the ML topic and a move beyond exploration toward more engaged implementation. This is a trend reinforced by the ML and artificial intelligence surveys we’ve run.
  • A 5% increase in usage for business-related material on the platform highlights the importance of tech for every facet of a business. It also aligns with the idea that all companies are now tech companies.
  • Security content went up 6% in usage in 2018, which is a good sign since we’ve noted in the past that security was underappreciated. Increased scrutiny from notable breaches may partly explain the increase. The development of distributed systems also presents new security challenges organizations must confront.
  • Web and mobile topics showed slight but noticeable declines in search and usage. We think the decline relates to maturity and a semantic transition. Organizations no longer pursue “web” and “mobile” computing; the web and mobile are now endemic enough that it’s all just “computing.”

Looking ahead

The rise of the Next Architecture, the maturation of blockchain, and emerging patterns in programming languages are areas of focus for us in the year ahead. We’ll continue to examine search and usage data on the platform, and we’ll also engage in research via conversations with our conference speakers and attendees, through perspectives from our community of practitioners and thought leaders, from media coverage, and from other sources. Ultimately, we want to see if these additional signals reinforce or challenge the findings from our platform data.