HT-ZF9 API and discovery process



  • Hi. You need to escape the quote marks on the MAN line. i.e.

    String customQuery = "M-SEARCH * HTTP/1.1\r\n" 
        + "HOST: 239.255.255.250:1900\r\n" 
        + "MAN: \"ssdp:discover\"\r\n"
        + "MX: 1\r\n"
        + "ST: ssdp:all\r\n" 
        + "\r\n";
    
    

    That would most likely explain why the M-SEARCH received no responses. You might like to try using the ST line recommended by Sony now, instead of "ssdp:all", to see if that works.



  • @grolschie it's already the case, it was removed by the copy and paste, sorry. With this query i can see my other upnp equipment...so i guess my model does not support it.



  • If you have an Android device, I recommend an app called UPnP Tool. It finds UPnP devices, even my Sony AVR's API service (when I cannot!). You might find something else to query, like I did.

    What character encoding are you encoding your String as, UTF-8?



  • I'm wondering if anyone can point us to some example code for the implementation of M-SEARCH discovery process please? It would be handy to know if there's some extra massaging required for the Sony devices to respond. I hit a road block here and never solved it.



  • @grolschie
    Your tool is working ! Here is a screenshot :
    HT-ZF9

    I don't know for UTF-8, I'm using this upnp api: https://github.com/custanator/android-upnp-discovery



  • It's not my tool. I wish it was, because then I'd know how to get my Sony AVR to respond to me properly. These Sony's are very picky at responding to M-SEARCH requests I think. 😞

    I looked at the the github project, and it's doing something similar to what I am doing and with the same results as me most likely. Maybe it is some Java weirdness? Here is the code in question.

    Maybe some Sony person can comment? Perhaps @david please?



  • Unfortunately I have the same problem as you, the devices are very picky then it comes to respond to discovery.
    Have had to use Wireshark to compare my messages to another application that worked to get it to work, and compare messages on bit level.
    I usually do the same way as @grolschie and use "ssdp:all" since the devices seems more likely to reply to that.
    Have also found that the devices in some cases can be very slow to respond so I had to use a longer timeout than expected and some times it take multiple tries.



  • @david Thanks for the reply. Is there any way engineers can update existing products to fix this? Or to get an explanation from them of what magic is actually required to get a response from the device (the official way)?



  • @grolschie I sort of got the normal engineering answer "it works for me".

    By the way here are my python script for discovery (haven't runned it for a while and think this version only works with python2). I'm using "ssdp:all" here since I was using it to test DLNA functions. I'm looking for both 'AVTransport', 'ContentDirectory' and 'ScalarWebAPI' functionality in the responce.

    #!/usr/bin/env python
    # encoding: UTF-8
    
    import socket
    import urllib2
    import urlparse
    import xml.etree.ElementTree as ET
    import StringIO
    
    SSDP_BROADCAST_ADDR = '239.255.255.250'
    SSDP_BROADCAST_PORT = 1900
    SSDP_BROADCAST_PARAMS = [
        'M-SEARCH * HTTP/1.1',
        "HOST: {0}:{1}".format(SSDP_BROADCAST_ADDR, SSDP_BROADCAST_PORT),
        'MAN: "ssdp:discover"', 'MX: 10', 'ST: ssdp:all', '', '']
    SSDP_BROADCAST_MSG = '\r\n'.join(SSDP_BROADCAST_PARAMS)
    
    def get_devices(timeout=10.0):
        """Find UPNP devices in current local network"""
        s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP)
        s.setsockopt(socket.IPPROTO_IP, socket.IP_MULTICAST_TTL, 4)
        s.bind(("", SSDP_BROADCAST_PORT + 10))
    
        s.sendto(SSDP_BROADCAST_MSG.encode('UTF-8'), (SSDP_BROADCAST_ADDR,
                                                      SSDP_BROADCAST_PORT))
    
        s.settimeout(timeout)
    
        devices = []
        while True:
            try:
                data, _addr = s.recvfrom(1024)
            except socket.timeout:
                break
    
            try:
                info = [a.split(":", 1)
                        for a in data.decode('UTF-8').split('\r\n')[1:]]
                device = dict([(a[0].strip().lower(), a[1].strip())
                               for a in info if len(a) >= 2])
                devices.append(device)
            except Exception:
                pass
    
        devices_urls = [{'location':dev['location'], 'type':dev['st']}
            for dev in devices if any(x in dev['st']
                for x in ['AVTransport', 'ContentDirectory', 'ScalarWebAPI'])]
        devices = [register_device(location_url) for location_url in devices_urls]
    
        return devices
    
    def register_device(location_url):
        """Get device information"""
        device_xml_url = location_url['location']
    
        print "Get device xml from {0}".format(device_xml_url)
    
        device_xml = urllib2.urlopen(device_xml_url).read().decode('UTF-8')
    
        info = parse_xml(device_xml)
    
        location = urlparse.urlparse(device_xml_url)
        hostname = location.hostname
    
        friendly_name = info.find('./device/friendlyName').text
    
        if 'ScalarWebAPI' in location_url['type']:
            xpath = './device/X_ScalarWebAPI_DeviceInfo/X_ScalarWebAPI_BaseURL'
            action_url = info.find(xpath).text
        else:
            xpath = "./device/serviceList/service/[serviceType='{0}']/controlURL"
            path = info.find(xpath.format(location_url['type'])).text
            action_url = urlparse.urljoin(location_url['location'], path)
    
        uuid = info.find('./device/UDN').text
    
        device = {
            'location': location_url['location'],
            'hostname': hostname,
            'friendly_name': friendly_name,
            'action_url': action_url,
            'st': location_url['type'],
            'uuid':uuid
        }
        return device
    
    def parse_xml(xml_string):
        """Removes namespaces and parse xml"""
        it = ET.iterparse(StringIO.StringIO(xml_string))
        for _, el in it:
            if '}' in el.tag:
                el.tag = el.tag.split('}', 1)[1]  # strip all namespaces
        return it.root
    


  • Thank you, @david.



  • I think I have found the github source for UPnP Tool:
    https://github.com/bjtj/upnp-java
    (It seems to be the same author)

    I will get it a look later



  • @TheFC-Company great find! I will have a more thorough look later. I'm no expert on datagrams, but I'm wondering if it has something to do with joining a multicast group vs merely sending a datagram to a multicast address - the latter is what I am doing and numerous other devices do respond.

    Edit: the project also uses a DatagramChannel, which is something I'm unfamiliar with.



  • @grolschie
    I have sent an email to the UPnP Tool guy, he confirm for the github code, here is it's full reply:

    M-SEARCH packet sent from UPnP Tool is like this and you can also check it or more packet logs by clicking "LOG…” Button on the main screen
    https://i.ibb.co/m59w5S3/upnp-tool.jpg
    If you need a java code sending M-SEARCH via datagram socket, please check this code
    https://github.com/bjtj/upnp-java/blob/master/src/main/java/com/tjapp/upnp/SSDPMsearchSender.java
    Above code is mine and very similar to UPnP Tool’s code

    πŸ™‚
    So the code that he use is:

    String customQuery = "M-SEARCH * HTTP/1.1\r\n" 
        + "HOST: 239.255.255.250:1900\r\n" 
        + "MAN: \"ssdp:discover\"\r\n"
        + "MX: 3\r\n"
        + "ST: upnp:rootdevice\r\n" 
        + "USER-AGENT: Android/27 UPnP/1.1 UPnPTool/1.5.1"
        + "\r\n";
    

    Unfortunately, that didn't help me to find the device. So we need to dig more in the datagram thing, but I'm not familiar too 😞



  • @TheFC-Company Thanks for the update. I also see that he uses an HttpHeader object rather than just construct a String. Then puts that into a byte array. I wonder why? πŸ€”



  • @TheFC-Company any joy? I have had a further look. So the format of that project's SSDP packet is pretty much the same. The HttpHeader object is just a class in the same project that stores key-value pairs.

    Functionally, the only major difference I can see is that he is using a DatagramChannel instead of a DatagramSocket. Hmmm.....



  • I found and fixed a bug in my detection code. I managed to finally get a response from the receiver using the Android emulator, but not from my actual phone. So for now "ssdp:all" shall remain my ST field value.



  • Hi @david. The weirdest thing. Using "ST: urn:schemas-sony.com:service:ScalarWebAPI:1" is now strangely working for me!!!??? I have no idea why! My original search string works (posted back on StackOverflow)!

    I'm wondering if the most recent firmware update for the STR-DN1080 changed something. I'm reluctant to roll out the corrected code in case it actually did and some users haven't updated their firmware. The release notes didn't mention anything.

    Failing that, a library update at my end or something...



  • @grolschie glad to hear that it is working for you now. Don't know if anything has changed in the last firmware and have limited access to devices right now, but tested on SRS-ZR5 and HT-ZF9 and "urn:schemas-sony-com:service:ScalarWebAPI:1" works for me too (had to resend the query a few times to be sure to get a response). Feels like I get a better response rate now than before, think I only got it to work reliably with "ssdp:all" before, but that is just a feeling...



  • @david thanks. Sometimes with UDP you may need to send 2 packets in quick succession, because delivery is not guaranteed. That's what I'm doing (I wasn't previously). Also I'm now sending on a different thread to listening for the reply. In doing so, now the socket starts listening for a reply before the first packet gets sent. So no replies should be missed.



  • @david You'll never believe this. But today it's not working again! I give up. Back to ssdp:all.
    ...or maybe not.

    Discovery:
    Interestingly, if you replace "ssdp" with any word you get even better results than ssdp:all (i.e. less devices responding), e.g. "ST: blah:all\r\n". And the Sony STR-DN1080's DIAL service does respond to this. LOL! This makes processing the results a lot quicker, especially if you have a lot of devices responding to ssdp:all and you are querying every device's device-desc.xml file for the model name. This might help someone.

    Edit: actually ST: urn:dial-multiscreen-org:device:dial:1 might be safer.