Tom Waits loves his trash. He takes journalists to rummage through junkyards during interviews. He drags bizarre detritus home to his family. He poses for photos amid heaps of dusty debris and populates his music with instruments made from refuse. But as charming as that may be, the last thing that anybody wants is a record full of garbage, let alone three of them, and a Waits career-retrospective box set could have been a tough slog. On first glance, Orphans—with its three discs divided by weird anthems (“Brawlers”), sad songs (“Bawlers”), and B-sides and assorted musical castoffs (“Bastards”)—looks like a typical Waitsian stroll through the junkyard. But his affection for trash turns what might have been a set of crappy fans-only throwaways into a record that puts the elusive singer-songwriter’s long career into manageable perspective—not least because there are only 14 old recordings here.

Waits’ best songs have always been crafted from rubbish—the human kind. Over the past three decades, he’s lovingly looted the lives of lowlifes and bottom feeders for their joys, sorrows, and redemptions. On his 1973 debut, Closing Time, Waits cast himself as a beat-inspired piano balladeer, pairing tales of drunken romance in all-night greasy spoons with Tin Pan Alley melodies. These were songs about blue-collar Joes remembering the good old days and looking at the future through the bottom of a bottle. Needless to say, in the era of studio-tanned California cocaine pop, this wasn’t the hottest gig in town. Most of Waits’ early success came from covers of his songs by artists like the Eagles, who took Waits’ song “Ol’ 55” and refitted it to sail on the smooth waters of yacht rock.

It was a dangerous time for anyone not named Billy Joel to be hanging out in a piano lounge. 1978’s show-tunes-y, synth-ridden Blue Valentine sanded the edge off Waits’ act, and he knew it: Working alongside his songwriting partner and wife Kathleen Brennan, Waits swapped record labels and went searching for weirder inspirations. He found them in such folks as Captain Beefheart, Kurt Weill, and Harry Partch—a Depression-era composer who spent his youth as a train-hopping hobo and wrote most of his music for instruments that included giant hanging glass balls. On subsequent records such as Swordfishtrombones and Rain Dogs, Waits reinvented himself as an offbeat poet, swinging wildly from heartfelt balladry to freakish South Seas stomps that swapped guitars for primitively instrumented thump. And still, most of his popular exposure came from covers by artists like Rod Stewart, who recast Waits’ bluesy “Downtown Train” as anthemic soft rock.

So it’s interesting that some of the most remarkable tracks on Orphans aren’t Waits originals. On his take on Lead Belly’s “Ain’t Goin’ Down to the Well” (a Brawler), an unsteady tambourine keeps time as Waits does rhythm fills with his voice, coughing the lines “Ain’t going down/Mama to the well/Mama to the well…no more” like Cookie Monster on an oxygen machine. There are no guitar solos, just amped-up spiritual uncertainty and sin. “Lie to Me” (another Brawler) finds Waits paying homage to Screamin’ Jay Hawkins. Here Waits’ voice quivers with bluesy swagger and rare jittery energy as he begs his lover to “Slap me baby/Give me your grief/I have no use for the truth.”

One of the reasons Waits has always stolen from the other side of the tracks is that he ascribes the residents there more soul. He manages to wring an abundance of that quality even from a three-minute bubblegum-punk blowout such as the Ramones’ “Danny Says” (a Bawler), on which Waits strips back the original’s hyperactive energy and snottiness like so much old paint, revealing a melancholy meditation about the boredom and loneliness of life on tour. “Soundcheck’s at 5:02/Record stores and interviews/But I can’t wait to be with you tomorrow,” Waits croons wearily.

Many of the songs on the “Bastards” disc are like broken toys—playful but slanted and tragic. Waits’ take on Daniel Johnston’s “King Kong” hits bipolar chemistry dead on. You can feel the spittle fly as Waits beatboxes his own chugging rhythm track. “When he saw the woman/He took her without question/Because after all he was the king/He loved the woman,” he hollers, letting the recap of the movie’s plot key in on Johnston’s gonzo innocence. It’s unsightly but sort of fun.

Waits also reclaims songs covered by others, such as “Down There by the Train,” a song that sounds like it could have been plucked off of Closing Time. It’s an epic weeper where Waits throws a brick on the piano’s sustain pedal and turns on the waterworks with lyrics like “There’s a place I know/Where the train goes slow/Where sinners can get washed in the blood of the lamb.” You can see why the tune appealed to Johnny Cash, who recorded it for his comeback album, American Recordings—it’s a regular cascade of tearful sentiment as Waits reads off the biggest names from the loser cannon. “I saw Judas Iscariot carrying John Wilkes Booth/Down there by the train,” he bellows. “First Kiss” is Waits peddling haywire hobo-Americana in his best spoken-word voice. “She cut two holes in the back of her dress, and she had these scapular wings that were covered with feathers and electrical tape,” he narrates, describing his inaugural smooch over rattletrap percussion and electronic samples.

“First Kiss” is what the “Bastards” disc was designed for—a catchall for oddballs that defy categorization—even to the detriment of what’s arguably the mission of a box set: culling a single timeless statement from a long career. But on the seven-minute anti-war song “The Road to Peace” (a Brawler), Waits isn’t timeless at all. He’s topical. Blues riffs ricochet like stray bullets while Waits details the injustices of the IsraelnPalestine conflict in a manner so direct that he might as well be reading Newsweek aloud. “Then the thunderous explosion blew out windows 200 yards away/With more retribution and 17 dead along the road to peace,” he drones in what is hopefully his final attempt at blues-journalism.

And some songs are merely trash. Waits’ second Lead Belly cover—an overproduced take on “Goodnight Irene” (a Bawler)—is so drunkenly shambolic that it teeters dangerously close to the edge of self-parody. On it, he indulges his worst show-tunes tendencies, bellowing the lyrics as a chorus of overdubbed Waits voices repeatedly holler “everybody!” “Dog Door” (a Bastard), a collaboration with Sparklehorse’s Mark Linkous, might also have been better off left at the curb. The song’s lurching trip-hop rhythms already sound more dated than the most ancient Waits material. “You got me coming through the dog door,” Waits hollers ad nauseam over a track that sounds like Massive Attack taking a whack at the Fat Possum back catalog.

But what’s a junkyard without some junk? On Orphans, Tom Waits supplies the musical equivalent of his beloved treasure hunts—a three-disc career-spanning boneyard where forlorn beauties poke through the rubbish. You’ve got to pick through crap like “The Road to Peace” in order to find a keeper like “Long Way Home” (a Bawler). “I know that I said I’d never do it again/And I love you, pretty baby/But I always take the long way home,” sings Waits on the hopeful country song, begging you to forget all those bad choices and just follow him out past the party lights. It’s a ballad that’s worth passing through a thousand landfills—or at least one “Dog Door”—for.

A good Tom Waits record has always been a product of his love affair with that which has been discarded. Orphans is a good Tom Waits record because the sprawling set—featuring cabaret creepers, gay-hobo weirdness, and clamoring garage-rock stompers—welds together all of the mismatched personalities and influences that Waits has been digging out of rummage bins over the years. Waits’ songs’ greatest strength is the palpable joy that he takes from their inspirations, and they shimmer with the songwriter’s care and attention despite their dubious origins.