Implementing "Network Idle" Wait without WebDriver BiDi #194

hanamichi77777 · 2026-03-01T00:44:48Z

hanamichi77777
Mar 1, 2026

This code utilizes a technique known as "Network Idle" to accurately wait for asynchronous browser processes—specifically communications via the Fetch API—to fully complete.

Instead of relying on a fixed pause (Sleep), it monitors the start and end of network requests on the JavaScript side. Control is only returned to VBA once a specified period of silence (e.g., 500ms with no active requests) has been detected.I think this is a very useful method when you aren't using bidirectional communication via WebDriver BiDi.

Key Points of the Wait Logic

Request Monitoring via Hooks (InstallNetworkAndDomHooks)
By overriding window.fetch and XMLHttpRequest, the script creates a global object (window.__niw) that maintains a counter of currently active requests. This makes the internal state of the browser transparent and trackable from the VBA environment.
Efficient Waiting via ExecuteScriptAsync
The use of driver.ExecuteScriptAsync within WaitForIdleStateAsync is a critical design choice:
Asynchronous Execution: A timer runs within the browser context, blocking the VBA execution until the specific condition (s.req === 0) is met.
Timeout Control: By implementing maxTimeToWait (30 seconds), the script ensures that VBA won't hang in an infinite loop if a network request fails to close or the connection remains stalled.
Handling the 5-Second Delay
While the HTML script calls https://httpbin.org/delay/5 (a service that intentionally delays responses for 5 seconds), the execution flows as follows:
Trigger: SelectByValue initiates the network request.
Wait Initiation: WaitForIdleStateAsync is called, putting the VBA thread into a waiting state.
Monitoring: As long as the browser is in a "Requesting" state (req=1), the callback cb('ok') is not triggered.

Completion Judgment: After 5 seconds, the Fetch request completes and req drops to 0. Once the idleTimeout (0.5s) passes without new activity, the "Stable" state is confirmed, and control returns to VBA.

Public Sub test_Classic_wait()
    Dim driver As New WebDriver
    Dim elem As WebElement

    driver.StartChrome
    driver.OpenBrowser
    
    Dim html As String
    html = "<!DOCTYPE html><html lang=""en""><head><meta charset=""UTF-8""><title>BiDi Tester</title><style>body{font-family:sans-serif;padding:20px;line-height:1.6}.container{border:1px solid #333;padding:20px;width:450px;background:#fff}label{font-weight:bold;display:block;margin-bottom:5px}select{width:100%;padding:8px;margin-bottom:20px}input{width:100%;padding:10px;border:1px solid #ccc;font-size:1em;color:#0056b3;background:#f0f8ff}.note{font-size:.85em;color:#666;margin-top:15px;border-top:1px dashed #ccc;padding-top:10px}</style></head>"
    html = html & "<body><div class=""container""><h3>Server-Side Delay Tester (5s)</h3><p>Proves BiDi can track long-running network requests.</p><label for=""userSelector"">Select Action:</label><select id=""userSelector"" onchange=""triggerLongFetch()""><option value="""">-- Choose to Trigger --</option><option value=""1"">Start 5-Second Server Request</option></select>"
    html = html & "<label for=""nameField"">Network/DOM Status:</label><input type=""text"" id=""nameField"" readonly placeholder=""Idle...""><div class=""note""><b>Mechanism:</b><br>1. Immediate DOM change (0ms).<br>2. Fetch starts immediately and stays open for <b>5s</b> using httpbin.org.<br>3. Final update occurs only after the server responds.</div></div>"
    html = html & "<script>function triggerLongFetch(){const s=document.getElementById('userSelector'),n=document.getElementById('nameField');if(s.value==="""")return;n.value=""Requesting... (Connection Open for 5s)"";fetch('https://httpbin.org/delay/5').then(r=>{if(!r.ok)throw new Error('Network error');return r.json()}).then(d=>{n.value=""Success: Response Received!""}).catch(e=>{n.value=""Error: Server unreachable or Timeout."";console.error(e)})}</script></body></html>"
    
    driver.NavigateToString html

    InstallNetworkAndDomHooks driver
    driver.FindElementByXPath("//select[@id='userSelector']").SelectByValue "1"
    WaitForIdleStateAsync driver

    Set elem = driver.FindElementByXPath("//input[@id='nameField']")
    Debug.Assert elem.GetProperty("value") = "Success: Response Received!"

    driver.CloseBrowser
    driver.Shutdown
End Sub

Private Sub InstallNetworkAndDomHooks(oDriver As WebDriver)
    Dim js As String
    js = ""

    ' Guard & state
    js = js & "if(!window.__niw){(function(){"
    js = js & "var s={req:0,lastMut:Date.now(),of:window.fetch,oo:XMLHttpRequest.prototype.open,os:XMLHttpRequest.prototype.send,onchange:null};"
    js = js & "function markChanged(){ s.lastMut=Date.now(); if(s.onchange) s.onchange(); }"

    ' fetch hook
    js = js & "try{window.fetch=function(){ s.req++; if(s.onchange) s.onchange(); return s.of.apply(this,arguments).finally(function(){ s.req--; if(s.onchange) s.onchange(); }); };}catch(e){}"

    ' XHR hook
    js = js & "try{XMLHttpRequest.prototype.open=function(){ return s.oo.apply(this,arguments); }; XMLHttpRequest.prototype.send=function(){ s.req++; if(s.onchange) s.onchange(); this.addEventListener('loadend',function(){ s.req--; if(s.onchange) s.onchange(); }); return s.os.apply(this,arguments); };}catch(e){}"

    ' MutationObserver
    js = js & "try{ new MutationObserver(function(){ markChanged(); }).observe(document,{childList:true,subtree:true,attributes:true}); }catch(e){}"

    ' value setter: input
    js = js & "try{ var d1=Object.getOwnPropertyDescriptor(HTMLInputElement.prototype,'value'); if(d1&&d1.configurable){ Object.defineProperty(HTMLInputElement.prototype,'value',{ get:d1.get, set:function(v){ markChanged(); return d1.set.call(this,v); } }); } }catch(e){}"

    ' value setter: textarea
    js = js & "try{ var d2=Object.getOwnPropertyDescriptor(HTMLTextAreaElement.prototype,'value'); if(d2&&d2.configurable){ Object.defineProperty(HTMLTextAreaElement.prototype,'value',{ get:d2.get, set:function(v){ markChanged(); return d2.set.call(this,v); } }); } }catch(e){}"

    ' expose
    js = js & "window.__niw=s;})();}"

    oDriver.ExecuteScript js
End Sub

Private Sub WaitForIdleStateAsync( _
    oDriver As WebDriver, _
    Optional idleTimeout As Long = 1000, _
    Optional maxTimeToWait As Long = 30000)

    Dim js As String, ret As Variant
    js = ""

    ' Receive arguments: idle quiet-window (ms), max overall timeout (ms), async callback
    js = js & "var idleTimeout=arguments[0], maxTimeout=arguments[1], cb=arguments[2];"

    ' Guard: hooks must be installed (__niw) or exit early
    js = js & "var s=window.__niw; if(!s){ cb('not-armed'); return; }"

    ' Timers: tm=quiet-window timer, safety=global timeout; settled prevents double completion
    js = js & "var tm=null, settled=false, safety=setTimeout(function(){ done('timeout'); }, maxTimeout);"

    ' Finish: ensure single completion, cleanup timers, then callback
    js = js & "function done(v){ if(settled) return; settled=true; if(tm) clearTimeout(tm); clearTimeout(safety); cb(v); }"

    ' Final re-check: verify conditions right before resolving to avoid races
    js = js & "function tryResolve(){ var quietFor=Date.now()-s.lastMut; if(s.req===0 && quietFor>=idleTimeout){ done('ok'); } else { schedule(); } }"

    ' Scheduler: when no network is in-flight (req==0), wait remaining quiet time then re-check
    js = js & "function schedule(){ if(tm) clearTimeout(tm); var quietFor=Date.now()-s.lastMut; if(s.req===0){ var remain=Math.max(0,idleTimeout-quietFor); var delay=Math.max(50,remain); tm=setTimeout(tryResolve,delay); } }"

    ' Subscribe to changes; defer the initial check by one tick to avoid s.req++ races
    js = js & "s.onchange=schedule; setTimeout(schedule,0);"

    ret = oDriver.ExecuteScriptAsync(js, idleTimeout, maxTimeToWait)
    If ret <> "ok" Then Err.Raise 404, , "Wait failed or timed out: " & ret
End Sub

hanamichi77777 · 2026-03-01T03:07:29Z

hanamichi77777
Mar 1, 2026
Author

I overlooked including DOM mutations in the monitoring scope, so I have revised the code as follows.

Note: As a control experiment,, commenting out the following two lines will prevent the driver from waiting until the process completes.

InstallNetworkAndDomHooks driver
WaitForIdleStateAsync driver, 600

note:Analyzing the JavaScript in this example reveals a 500ms delay after 'Done!' is displayed. Therefore, I have set the idle timeout to 600ms—slightly longer than the 500ms threshold—to ensure proper synchronization.Refer to the next comment for instructions on how to configure this.

Example Page:
https://www.selenium.dev/selenium/web/ajaxy_page.html

Public Sub test_Classic_wait2()
    Dim driver As New WebDriver
    Dim elem As WebElement

    driver.StartChrome
    driver.OpenBrowser
    
    driver.NavigateTo "https://www.selenium.dev/selenium/web/ajaxy_page.html"

    driver.FindElement(By.xpath, "//input[@name='typer']").SendKeys "aaa"
    driver.FindElement(By.xpath, "//input[@id='red']").Click
    
    InstallNetworkAndDomHooks driver
    driver.FindElement(By.XPath,"//input[@value='Add Label']").Click
    WaitForIdleStateAsync driver, 600

    Debug.Assert driver.FindElement(By.xpath, "//div[@id='update_butter']").GetText = "Done!"
    driver.CloseBrowser
    driver.Shutdown
End Sub

Private Sub InstallNetworkAndDomHooks(oDriver As WebDriver)
    Dim js As String
    js = ""

    ' Guard & state
    js = js & "if(!window.__niw){(function(){"
    js = js & "var s={req:0,lastMut:Date.now(),of:window.fetch,oo:XMLHttpRequest.prototype.open,os:XMLHttpRequest.prototype.send,onchange:null};"
    js = js & "function markChanged(){ s.lastMut=Date.now(); if(s.onchange) s.onchange(); }"

    ' fetch hook
    js = js & "try{window.fetch=function(){ s.req++; if(s.onchange) s.onchange(); return s.of.apply(this,arguments).finally(function(){ s.req--; if(s.onchange) s.onchange(); }); };}catch(e){}"

    ' XHR hook
    js = js & "try{XMLHttpRequest.prototype.open=function(){ return s.oo.apply(this,arguments); }; XMLHttpRequest.prototype.send=function(){ s.req++; if(s.onchange) s.onchange(); this.addEventListener('loadend',function(){ s.req--; if(s.onchange) s.onchange(); }); return s.os.apply(this,arguments); };}catch(e){}"

    ' MutationObserver
    js = js & "try{ new MutationObserver(function(){ markChanged(); }).observe(document,{childList:true,subtree:true,attributes:true}); }catch(e){}"

    ' value setter: input
    js = js & "try{ var d1=Object.getOwnPropertyDescriptor(HTMLInputElement.prototype,'value'); if(d1&&d1.configurable){ Object.defineProperty(HTMLInputElement.prototype,'value',{ get:d1.get, set:function(v){ markChanged(); return d1.set.call(this,v); } }); } }catch(e){}"

    ' value setter: textarea
    js = js & "try{ var d2=Object.getOwnPropertyDescriptor(HTMLTextAreaElement.prototype,'value'); if(d2&&d2.configurable){ Object.defineProperty(HTMLTextAreaElement.prototype,'value',{ get:d2.get, set:function(v){ markChanged(); return d2.set.call(this,v); } }); } }catch(e){}"

    ' expose
    js = js & "window.__niw=s;})();}"

    oDriver.ExecuteScript js
End Sub

Private Sub WaitForIdleStateAsync( _
    oDriver As WebDriver, _
    Optional idleTimeout As Long = 1000, _
    Optional maxTimeToWait As Long = 30000)

    Dim js As String, ret As Variant
    js = ""

    ' Receive arguments: idle quiet-window (ms), max overall timeout (ms), async callback
    js = js & "var idleTimeout=arguments[0], maxTimeout=arguments[1], cb=arguments[2];"

    ' Guard: hooks must be installed (__niw) or exit early
    js = js & "var s=window.__niw; if(!s){ cb('not-armed'); return; }"

    ' Timers: tm=quiet-window timer, safety=global timeout; settled prevents double completion
    js = js & "var tm=null, settled=false, safety=setTimeout(function(){ done('timeout'); }, maxTimeout);"

    ' Finish: ensure single completion, cleanup timers, then callback
    js = js & "function done(v){ if(settled) return; settled=true; if(tm) clearTimeout(tm); clearTimeout(safety); cb(v); }"

    ' Final re-check: verify conditions right before resolving to avoid races
    js = js & "function tryResolve(){ var quietFor=Date.now()-s.lastMut; if(s.req===0 && quietFor>=idleTimeout){ done('ok'); } else { schedule(); } }"

    ' Scheduler: when no network is in-flight (req==0), wait remaining quiet time then re-check
    js = js & "function schedule(){ if(tm) clearTimeout(tm); var quietFor=Date.now()-s.lastMut; if(s.req===0){ var remain=Math.max(0,idleTimeout-quietFor); var delay=Math.max(50,remain); tm=setTimeout(tryResolve,delay); } }"

    ' Subscribe to changes; defer the initial check by one tick to avoid s.req++ races
    js = js & "s.onchange=schedule; setTimeout(schedule,0);"

    ret = oDriver.ExecuteScriptAsync(js, idleTimeout, maxTimeToWait)
    If ret <> "ok" Then Err.Raise 404, , "Wait failed or timed out: " & ret
End Sub

0 replies

hanamichi77777 · 2026-03-01T03:31:14Z

hanamichi77777
Mar 1, 2026
Author

Even when using classic commands, BiDi can be leveraged as a wait strategy tool. Running the following code generates logs in the Immediate Window; by pasting these into an AI and asking 'What is the most efficient idle wait time?', you can obtain the optimal setting.


Public Sub test_BiDi_wait2()
    Dim driver As New WebDriver
    Dim elem As WebElement

    driver.StartChrome
    
   ' Browser startup settings
    Dim caps As SeleniumVBA.WebCapabilities: Set caps = driver.CreateCapabilities
    caps.EnableBiDiMode
    
    ' Open
    driver.OpenBrowser caps
    Dim bidi As New BiDiCommandWrapper: bidi.ConnectTo driver.GetWebSocketUrl

    driver.NavigateTo "https://www.selenium.dev/selenium/web/ajaxy_page.html"

    driver.FindElement(By.xpath, "//input[@name='typer']").SendKeys "aaa"
    driver.FindElement(By.xpath, "//input[@id='red']").Click
    
    bidi.ExecuteClickByXPath "//input[@value='Add Label']"

    Debug.Assert driver.FindElement(By.xpath, "//div[@id='update_butter']").GetText = "Done!"

    Set bidi = Nothing
    driver.CloseBrowser
    driver.Shutdown
End Sub

This is the .xlsm file containing the code mentioned above. The password is 123.
SeleniumVBA7.2_BiDi_Test_123_Waiting.xlsm

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implementing "Network Idle" Wait without WebDriver BiDi #194

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Replies: 2 comments

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Select a reply

Uh oh!

Implementing "Network Idle" Wait without WebDriver BiDi #194

Uh oh!

Uh oh!

hanamichi77777 Mar 1, 2026

Replies: 2 comments

Uh oh!

Uh oh!

hanamichi77777 Mar 1, 2026 Author

Uh oh!

Uh oh!

hanamichi77777 Mar 1, 2026 Author

hanamichi77777
Mar 1, 2026

hanamichi77777
Mar 1, 2026
Author

hanamichi77777
Mar 1, 2026
Author