Machine-learning system finds patterns in materials “recipes,” even when training data is lacking.


MIT Materials Synthesis Web
A new machine-learning system for analyzing materials “recipes” uses a variational autoencoder, which squeezes data (left-hand circles) down into a more compact form (center circles) before attempting to re-expand it into its original form (right-hand circles). If the autoencoder is successfully trained, the compact representation will capture the data’s most salient characteristics. Image, Chelsea Turner, MIT

Last month, three MIT materials scientists and their colleagues published a paper describing a new artificial-intelligence system that can pore through scientific papers and extract “recipes” for producing particular types of materials.
That work was envisioned as the first step toward a system that can originate recipes for materials that have been described only theoretically. Now, in a paper in the journal npj Computational Materials, the same three materials scientists, with a colleague in MIT’s Department of Electrical Engineering and Computer Science (EECS), take a further step in that direction, with a new artificial-intelligence system that can recognize higher-level patterns that are consistent across recipes.

For instance, the new system was able to identify correlations between “precursor” chemicals used in materials recipes and the crystal structures of the resulting products. The same correlations, it turned out, had been documented in the literature.

The system also relies on statistical methods that provide a natural mechanism for generating original recipes. In the paper, the researchers use this mechanism to suggest alternative recipes for known materials, and the suggestions accord well with real recipes.

The first author on the new paper is Edward Kim, a graduate student in materials science and engineering. The senior author is his advisor, Elsa Olivetti, the Atlantic Richfield Assistant Professor of Energy Studies in the Department of Materials Science and Engineering (DMSE). They’re joined by Kevin Huang, a postdoc in DMSE, and by Stefanie Jegelka, the X-Window Consortium Career Development Assistant Professor in EECS.

Sparse and scarce

Like many of the best-performing artificial-intelligence systems of the past 10 years, the MIT researchers’ new system is a so-called neural network, which learns to perform computational tasks by analyzing huge sets of training data. Traditionally, attempts to use neural networks to generate materials recipes have run up against two problems, which the researchers describe as sparsity and scarcity.

Any recipe for a material can be represented as a vector, which is essentially a long string of numbers. Each number represents a feature of the recipe, such as the concentration of a particular chemical, the solvent in which it’s dissolved, or the temperature at which a reaction takes place.

Since any given recipe will use only a few of the many chemicals and solvents described in the literature, most of those numbers will be zero. That’s what the researchers mean by “sparse.”

Similarly, to learn how modifying reaction parameters — such as chemical concentrations and temperatures — can affect final products, a system would ideally be trained on a huge number of examples in which those parameters are varied. But for some materials — particularly newer ones — the literature may contain only a few recipes. That’s scarcity.

“People think that with machine learning, you need a lot of data, and if it’s sparse, you need more data,” Kim says. “When you’re trying to focus on a very specific system, where you’re forced to use high-dimensional data but you don’t have a lot of it, can you still use these neural machine-learning techniques?”

Neural networks are typically arranged into layers, each consisting of thousands of simple processing units, or nodes. Each node is connected to several nodes in the layers above and below. Data is fed into the bottom layer, which manipulates it and passes it to the next layer, which manipulates it and passes it to the next, and so on. During training, the connections between nodes are constantly readjusted until the output of the final layer consistently approximates the result of some computation.

The problem with sparse, high-dimensional data is that for any given training example, most nodes in the bottom layer receive no data. It would take a prohibitively large training set to ensure that the network as a whole sees enough data to learn to make reliable generalizations.

Artificial bottleneck

The purpose of the MIT researchers’ network is to distill input vectors into much smaller vectors, all of whose numbers are meaningful for every input. To that end, the network has a middle layer with just a few nodes in it — only two, in some experiments.

The goal of training is simply to configure the network so that its output is as close as possible to its input. If training is successful, then the handful of nodes in the middle layer must somehow represent most of the information contained in the input vector, but in a much more compressed form. Such systems, in which the output attempts to match the input, are called “autoencoders.”

Autoencoding compensates for sparsity, but to handle scarcity, the researchers trained their network on not only recipes for producing particular materials, but also on recipes for producing very similar materials. They used three measures of similarity, one of which seeks to minimize the number of differences between materials — substituting, say, just one atom for another — while preserving crystal structure.

During training, the weight that the network gives example recipes varies according to their similarity scores.

Playing the odds

In fact, the researchers’ network is not just an autoencoder, but what’s called a variational autoencoder. That means that during training, the network is evaluated not only on how well its outputs match its inputs, but also on how well the values taken on by the middle layer accord with some statistical model — say, the familiar bell curve, or normal distribution. That is, across the whole training set, the values taken on by the middle layer should cluster around a central value and then taper off at a regular rate in all directions.

After training a variational autoencoder with a two-node middle layer on recipes for manganese dioxide and related compounds, the researchers constructed a two-dimensional map depicting the values that the two middle nodes took on for each example in the training set.
Remarkably, training examples that used the same precursor chemicals stuck to the same regions of the map, with sharp boundaries between regions. The same was true of training examples that yielded four of manganese dioxide’s common “polymorphs,” or crystal structures. And combining those two mappings indicated correlations between particular precursors and particular crystal structures.

“We thought it was cool that the regions were continuous,” Olivetti says, “because there’s no reason that that should necessarily be true.”

Variational autoencoding is also what enables the researchers’ system to generate new recipes. Because the values taken on by the middle layer adhere to a probability distribution, picking a value from that distribution at random is likely to yield a plausible recipe.

“This actually touches upon various topics that are currently of great interest in machine learning,” Jegelka says. “Learning with structured objects, allowing interpretability by and interaction with experts, and generating structured complex data — we integrate all of these.”

“‘Synthesizability’ is an example of a concept that is central to materials science yet lacks a good physics-based description,” says Bryce Meredig, founder and chief scientist at Citrine Informatics, a company that brings big-data and artificial-intelligence techniques to bear on materials science research. “As a result, computational screens for new materials have been hamstrung for many years by synthetic inaccessibility of the predicted materials. Olivetti and colleagues have taken a novel, data-driven approach to mapping materials syntheses and made an important contribution toward enabling us to computationally identify materials that not only have exciting properties but also can be made practically in the laboratory.”

The research was supported by the National Science Foundation, the Natural Sciences and Engineering Research Council of Canada, the U.S. Office of Naval Research, the MIT Energy Initiative, and the U.S. Department of Energy’s Basic Energy Science Program.

Larry Hardesty | MIT News Office
December 21, 2017



Scholars will engage in a year of postgraduate leadership studies at Beijing’s Tshingua University. 
MIT Schwarzman 0
Schwarzman Scholars from top left, clockwise: Katheryn Scott, Han Wu, Henry Aspegren, Joshua Woodard. Images courtesy of Schwarzman Scholars. Images courtesy of Schwarzman Scholars.

Three MIT students — Henry Aspegren '17, Katheryn Scott, and Joshua Woodard — were selected as Schwarzman Scholars and will begin postgraduate studies at Tsinghua University in Beijing next fall. An alumnus, Han Wu MEng '15, was also selected for this highly competitive program.

Schwarzman Scholars are chosen based on demonstrated leadership qualities and potential to bridge and understand cultural and political differences. They will live in Beijing for a year of study and cultural immersion, attending lectures, traveling, and developing a better understanding of China.

This year’s four Schwarzman Scholars bring to 11 the total number of MIT winners honored since the scholarship’s inception in 2015. In all, 142 Schwarzman Scholars were selected from over 4,000 applicants. The new class is comprised of students from 39 countries and 97 universities with 41 percent from the United States, 20 percent from China, and 39 percent from the rest of the world. The currently enrolled MIT students were supported by MIT’s Office of Distinguished Fellowships the Presidential Committee on Distinguished Fellowships.

“This year’s winners of the Schwarzman Scholarship exemplify the combination of intellectual prowess and public mindedness that characterizes MIT students at their best,” says Professor William Broadhead, co-chair of the Presidential Committee for Distinguished Fellowships alongside Professor Rebecca Saxe. “Those of us who have had the pleasure of working with them through the application process have been impressed at every turn by their immense potential for local and global leadership. It’s exciting to celebrate with them now; and it will be exciting to see what they do next!”

Henry Aspegren

Henry Aspegren, from Ann Arbor, Michigan, is an MIT master’s student in engineering. He received his BS in electrical engineering and computer science from MIT earlier this year. Aspegren aspires to develop public policy for addressing the new challenges and opportunities created by technology.

Aspegren recognized the economic disparities of the Detroit area growing up, when he played ice hockey on a team with players from manufacturing towns around metro Detroit that had been hit hard by the decline of the auto industry. This reality drew him to think about how economic incentives can stimulate economies, which fueled his academic interests in currency and financial institutions.

At MIT, Aspegren began conducting research in the MIT Media Lab’s Viral Communications Group, where he worked to help build a voting and ranking algorithm to quantify subjective qualities such as emotion across the internet in real time. During his junior year, he participated in the Cambridge MIT Exchange program and received a first from Cambridge University and a full blue in ice hockey.
This past January, Aspegren traveled to Korea through the MIT International Science and Technology Initiatives' Global Teaching Laboratory to lead a robotics workshop in which students programmed a Roomba vaccum cleaner to drive around an obstacle course. He has also interned with the electronic trading team at Goldman Sachs in New York and London, and worked as a software engineer with BetterWorks in Palo Alto.

Aspegren is now completing his MEng degree and conducting research with the MIT Media Lab’s Digital Currency Initiative to examine injustices in financing. This led him to design a block chain-based system for agricultural financing in Latin America in collaboration with the InterAmerican Development Bank.

Aspegren has been an active participant in MIT Athletics, playing club ice hockey throughout his undergraduate and graduate career, and playing on the varsity lacrosse team his freshman year. He is also a brother of Theta Chi Fraternity.

Katheryn Scott

Katheryn "Kate" Scott, from Barrington, Illinois, is an MIT senior majoring in materials science and engineering. She studied abroad at Oxford University in her junior year through the Department of Materials Science and Engineering’s exchange program. Scott seeks to pursue a future career bridging the gap between science and communications, and eventually plans to found her own communications firm.

In the summer of her freshman year, Scott traveled to Singapore to conduct materials research, fabricating thin-film membranes to create nano-filtration systems for smog. She later began research with the MIT Libraries Conservation Lab, prototyping two different devices for reversible flattening of manuscripts, which would automate part of the conservation process. At Oxford, Scott conducted polymer research with the Polymer Group and Ashmolean Museum.

Scott has a keen interest in industry, and worked as a chemical engineering intern at Honeywell UOP. While there, she worked to improve wastewater filtration by developing a disinfectant and low temperature tolerant bacteria. The system saves 400,000 gallons of wastewater per day, results that led to the adoption of her system in October 2016.

Scott is a sorority sister of Sigma Kappa, and has held the role of continuing membership chair and new member assistant coordinator. She was elected as vice president of programming for the MIT Panhellenic Association.
Since Scott’s freshman year, she has been a member of MIT’s only Division I sport, rowing. She and her boat earned a bid to the 2016 national competition, and placed 5th, and Scott was named a Collegiate Rowing Coaches Association Scholar Athlete. When she was at Oxford University, she joined the university’s lightweight rowing club.

Joshua Woodard

Joshua Charles Woodard, from Chicago, Illinois, is an MIT senior majoring in mechanical engineering with a minor in Mandarin Chinese. At Tsinghua, Woodard will earn a degree in politics, with a focus on comparative government. He plans a future career in diplomacy and public policy, with the goal of enacting effective strategies for social change.

Woodard’s dedication to social justice issues began prior to arriving at MIT. As a junior in high school, he applied for and was granted a Boeing Scholars Academy award to research Chicago’s gun violence and devise solutions. He then coordinated a city-wide brainstorming event between youth and government officials.

At MIT, Woodard has been a pivotal voice on issues of diversity and inclusion. As a student advisor on MIT President L. Rafael Reif’s Presidential Advisory Committee, he has provided guidance on important campus issues and policies ranging from diversity initiatives to the influence of the current political climate. Woodard has also demonstrated his leadership skills as co-chair of the student community and living group Chocolate City, and has been instrumental in increasing campus awareness of the Black Lives Matter movement and creating opportunities for dialogue.

Woodard participated in the Internationally Genetically Engineered Machines (iGEM) worldwide competition for synthetic biology, and he has interned in industrial design at the Charles Stark Draper Laboratory and HTC. He has also advocated to help local Boston high school students from underrepresented communities gain access to STEM experiences by co-founding the summer leadership program MIT BoSTEM Scholars Academy.

A talented artist and musician, Woodard has studied and performed Beijing Opera at the Shanghai Theater Academy in China, runs his own freelance photography business, JC Woodard Photography, and has performed on violin and viola with the MIT Jazz Band.

Han Wu

Han Wu graduated from MIT in 2015 with a master's degree in structural engineering focusing on high performance structures.

Prior to enrolling at MIT, he received a bachelor’s degree from the University of California at Los Angeles majoring in civil and environmental engineering and minoring in accounting. Currently, he works at Ove Arup and Partners Hong Kong (one of the worldwide leading engineering consulting firms) as a structural engineer and the chairman of Young Engineer’s Group.

Besides tackling challenging design problems, Wu also plays a key role in researching and implementing industry leading design tools as well as conducting training sessions. Upon completion of Schwarzman Scholars, he hopes to pursue a career in which he can combine his experience and knowledge in design and business development.

back to newsletter

Kim Benard | Office of Distinguished Fellowships
MIT News Office
December 4, 2017

BASF P380 D17 ScienceAward 001 Rupp Full Web
BASF and Volkswagen Science Award Electrochemistry presentation Dec. 1, 2017, at Karlsruhe Institute of Technology in Germany. Standing (l-r): Volkswagen AG Research and Development Group Head Ulrich Eichhorn, Catalytic Innovations founder and CEO Stafford Sheehan, who received a special prize for applied research MIT Assistant Professor of Materials Science and Engineering Jennifer Rupp, who received the Science Award Electrochemistry, BASF Chief Technology Officer and Vice Chairman of the Board of Executive Directors Martin Brudermüller, and Karlsruhe Institute of Technology President Holger Hanselka. BASF photo.

Jennifer L. M. Rupp, who holds joint appointments at MIT as an assistant professor in the Departments of Materials Science and Engineering [DMSE] and Electrical Engineering and Computer Science [EECS], won the 2017 “Science Award Electrochemistry,” awarded by Volkswagen and BASF. Rupp was honored for her work on energy storage systems.

Rupp received the award, which is worth about $47,000, on Dec. 1, 2017, at ceremonies held at Karlsruhe Institute of Technology (KIT) in Germany. Rupp’s Electrochemical Materials Laboratory at MIT is working to replace the flammable liquid electrolyte in lithium batteries with a safer solid-state lithium electrolyte.

“The team was honored to receive the award for their work on processing and designing new solid-state, garnet-type batteries and for their commitment to integrate cathodes with socio-economically acceptable elements," Rupp says. “Designing lithium conducting glass-ceramics and battery electrode alloys can be interesting strategies for future battery architectures based on garnets to avoid lithium dendrites that often lead to performance failure," Rupp says. Dendrites are lithium filaments shaped like tree leaves or snowflakes that can form in rechargeable lithium metal batteries, and their unchecked growth can cause a cell to short-circuit.

“The winners of our Science Award are an excellent example of innovative and creative ideas in this field,” says Dr. Ulrich Eichhorn, head of Group Research and Development for Volkswagen AG. The German automaker plans to reach a goal of 25 percent battery-powered electric vehicles by 2025.

The Science Award Electrochemistry noted Rupp’s work on ceramic engineering for fast lithium transfer in garnet-type batteries and a novel glassy-type lithium ion conductor that may lead to new design principles for solid-state batteries. “BASF creates chemistry for a sustainable future. We all know that batteries are at the core of electromobility, and there is great potential for specific technological progress in this area. Yet, there are scientific hurdles we must first overcome,” says Martin Brudermüller, Vice Chairman of the Board of Executive Directors and Chief Technology Officer at BASF. “Electrochemistry is a key technology for sustainable future mobility. That is why we need first-class research around the globe conducted by excellent scientists who inspire each other to continuously develop new and better solutions.”

Rupp joined the MIT Department of Materials Science and Engineering in January 2017 as the Thomas Lord Assistant Professor of Materials Science and Engineering at MIT, and recently was appointed as an assistant professor in the Department of Electrical Engineering and Computer Science. She also conducts research on materials for solid oxide fuel cells, electrochemical sensors and information storage devices.

The BASF and Volkswagen International “Science Award Electrochemistry” has been awarded yearly since 2012.

back to newsletter

– Materials Research Laboratory
December 19, 2017

More than half of Roxbury, Bunker Hill, students who get summer lab experience at MIT go on to earn a four-year degree.
Susan Rosevear Fall 2017 MRS 0541 DP Web
Community college students who experience a summer of research at MIT develop greater self-confidence and better academic skills, with a majority completing a four-year degree, MIT Materials Research Laboratory Education Officer Susan Rosevear told a symposium at the Materials Research Society Fall meeting in Boston on Monday, Nov. 27, 2017. Photo, Denis Paiste, MIT MRL

A summer of research at MIT gives inner-city Boston community college students a pathway toward greater self-confidence, better academic skills and a four-year college degree, MIT Materials Research Laboratory Education Officer Susan Rosevear said Monday, Nov. 27, 2017, during a symposium at the Materials Research Society [MRS] Fall meeting in Boston.

“Many of them have barely heard about materials science when they come to MIT, and by the end of the summer, they get sort of a full dunk into the world of materials science, so they are better informed to go forward,” Rosevear says. Over the past dozen years, 63 students from Roxbury and Bunker Hill Community Colleges have participated in the program at MIT. Of these, 45 went on to earn a four-year degree, with 34 pursuing degrees in science or engineering. Five continued on to graduate school in science or medicine.

The Research Experience for Undergraduates (REU) program is primarily funded through the MIT Materials Research Laboratory’s National Science Foundation-funded Materials Research Science and Engineering Center [NSF-MRSEC]. Bringing in underrepresented, or non-traditional, students from the community colleges broadens the diversity of students in the REU program.

“We are trying, and I think succeeding, in providing opportunities to community college students that they don’t have at their home institutions,” Rosevear says. Students learn to use electron microscopes, X-ray diffraction spectrometers and other advanced materials science characterization tools. Rosevear addressed a session at MRS highlighting collaborations between community colleges and four-year colleges.

In 2005, the MIT MRSEC, then part of the Center for Materials Science and Engineering, began the partnership with Roxbury Community College with seven students participating during its first year. In recent years, the summer program expanded to include community college professors in materials research on campus led by MIT faculty. So far, nine community college professors have participated. CMSE merged in October 2017 with the Materials Processing Center to form the MIT Materials Research Laboratory.

During the fall 2017 semester, Roxbury Community College Chemistry and Biotechnology Professor Kimberly Stieglitz offered a new course at Roxbury Community College, Research Science, [SCI 281] that brought students to the X-ray diffraction facility at MIT to examine their lab samples. “We keep finding new ways to leverage this partnership,” Rosevear says. Stieglitz and other teachers who have participated in the summer teachers’ program at MIT, also have incorporated material from their summer research into their classroom teaching, Rosevear notes.

Students must complete a basic engineering or science course, such as chemistry or biology, to be accepted into the MIT summer program. Community college teachers select the students based on academic record, statements of interest and faculty letters of recommendation. “They’ve been great partners for us, which is really key to the whole thing,” Rosevear explains. “Kimberly [Stieglitz] has told me, once they are selected, just knowing they are going to MIT changes their performance, they become more serious about themselves, their performance, motivation increases, and they have an increased commitment to STEM,” Rosevear says.

CMSE Scholar Kimberly Stieglitz Jode Lavine 9157 DP Web
Roxbury Community College Chemistry and Biotechnology Professor Kimberly Stieglitz [left] discusses her summer research at MIT with JoDe M. Lavine, Bunker Hill Community College Professor and Chairperson of the Engineering & Physical Sciences Department, during the annual Summer Scholars Poster Session on Aug. 3, 2017. Stieglitz worked in the lab of AMAX Career Development Assistant Professor in Materials Science and Engineering Robert J. Macfarlane. Photo, Denis Paiste, MIT MRL.

Over the course of the summer, community college students attend weekly luncheon meetings covering topics such as crafting a high-quality poster presentation, applying to graduate school, understanding patents and trademarks, and pursuing careers in materials science and other engineering fields.

Interest among MIT faculty in hosting community college students continues to grow. “I have people coming to me and say, how do I get one of these students?
The students have sold themselves, is essentially what’s happened,” Rosevear says.

The community college program is distinct from the Summer Scholars program, which is open to undergraduates in science and engineering from across the U.S. and Puerto Rico who are citizens or legal residents. Applications for summer 2018 must be submitted by Feb. 16, 2018.

 back to newsletter

Denis Paiste, MIT Materials Research Laboratory
December 19, 2017