It never actually occurred to me to try to figure it out until I read Elohim's characterization of himself vs. dead idols in Jer 4:1 

"O Israel, if you will truly return to me and absolutely discard your idols, 2 and if you will swear by me alone, the living God, and begin to live good, honest, clean lives, then you will be a testimony to the nations of the world, and they will come to me and glorify my name." TLB

Deut 5:26 What man can hear, as we have, the voice of the living God speaking from the heart of the fire, and live? You go and listen to all that God says, then come and tell us, and we will listen and obey.' TLB
Jer 10:10 But the Lord is the only true God, the living God, the everlasting King. The whole earth shall tremble at his anger; the world shall hide before his displeasure. TLB

Proof that He is a Living God

Josh 3:9 Then Joshua summoned all the people and told them, "Come and listen to what the Lord your God has said. 10 Today you are going to know for sure that the living God is among you and that he will, without fail, drive out the Canaanites, Hittites, Hivites, Perizzites, Girgashites, Amorites, and Jebusites-all the people who now live in the land you will soon occupy. TLB
Acts 14:15 "Men! What are you doing? We are merely human beings like yourselves! We have come to bring you the Good News that you are invited to turn from the worship of these foolish things and to pray instead to the living God, who made heaven and earth and sea and everything in them. 16 In bygone days he permitted the nations to go their own ways, 17 but he never left himself without a witness; there were always his reminders-the kind things he did such as sending you rain and good crops and giving you food and gladness." TLB

Sovereign to do as He pleases

Job 27:2 "I vow by the living God, who has taken away my rights, even the Almighty God who has embittered my soul, TLB

Because He's alive we need to meet Him personally for salvation

Ps 42:1 As the deer pants for water, so I long for you, O God. 2 I thirst for God, the living God. Where can I find him to come and stand before him? TLB
1 Tim 4:9 This is the truth and everyone should accept it. We work hard and suffer much in order that people will believe it, for our hope is in the living God who died for all, and particularly for those who have accepted his salvation. TLB
Heb 9:14 just think how much more surely the blood of Christ will transform our lives and hearts. His sacrifice frees us from the worry of having to obey the old rules and makes us want to serve the living God. For by the help of the eternal Holy Spirit, Christ willingly gave himself to God to die for our sins-he being perfect, without a single sin or fault. TLB

Dangerous to blaspheme the living God

Isa 37:4 But perhaps the Lord your God heard the blasphemy of the king of Assyria's representative as he scoffed at the living God. Surely God won't let him get away with this. TLB

Others also recognized His reputation as the Living God

Dan 6:19 Very early the next morning he hurried out to the lions' den 20 and called out in anguish, "O Daniel, servant of the Living God, was your God, whom you worship continually, able to deliver you from the lions?" TLB

Israelis are sons, children of the Living God

Hos 1:10 "Yet the time will come when Israel shall prosper and become a great nation; in that day her people will be too numerous to count-like sand along a seashore! Then, instead of saying to them, 'You are not my people,' I will tell them, 'You are my sons, children of the Living God.' TLB (Deut 14:1 You are the children of the LORD your God. Do not cut yourselves or shave the front of your heads for the dead, 2 for you are a people holy to the LORD your God. Out of all the peoples on the face of the earth, the LORD has chosen you to be his treasured possession. NIV )
Jer 31:9 because I am Israel's father, and Ephraim is my firstborn son. NIV
Isa 63:8 He said, "Surely they are my people, sons who will not be false to me"; and so he became their Savior. NIV

Rom 9:25 Remember what the prophecy of Hosea says? There God says that he will find other children for himself (who are not from his Jewish family) and will love them, though no one had ever loved them before. 26 And the heathen, of whom it once was said, "You are not my people," shall be called "sons of the Living God." TLB <>

Messiah is also a Son of the living God

Matt 16:15 Then he asked them, "Who do you think I am?" 16 Simon Peter answered, "The Christ, the Messiah, the Son of the living God." 17 "God has blessed you, Simon, son of Jonah," Jesus said, "for my Father in heaven has personally revealed this to you-this is not from any human source. TLB
Matt 26:63 Then the High Priest said to him, "I demand in the name of the living God that you tell us whether you claim to be the Messiah, the Son of God." 64 "Yes," Jesus said, "I am. And in the future you will see me, the Messiah, sitting at the right hand of God and returning on the clouds of heaven." TLB

The Word is alive because it's from the Spirit of the living God

2 Cor 3:3 They can see that you are a letter from Christ, written by us. It is not a letter written with pen and ink, but by the Spirit of the living God; not one carved on stone, but in human hearts. TLB

Dead Idols of gods have filth (demons) attached to them

2 Cor 6:16 And what union can there be between God's temple and idols? For you are God's temple, the home of the living God, and God has said of you, "I will live in them and walk among them, and I will be their God and they shall be my people." 17 That is why the Lord has said, "Leave them; separate yourselves from them; don't touch their filthy things, and I will welcome you 18 and be a Father to you, and you will be my sons and daughters." TLB